[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:22:42 dist-test-slave-4d5t krb5kdc[14911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:22:42 dist-test-slave-4d5t krb5kdc[14911](info): set up 2 sockets
Sep 26 21:22:42 dist-test-slave-4d5t krb5kdc[14911](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 21:22:43.141392 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:22:43 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:22:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:22:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:22:43.329291 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46377
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:43.684724 14924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:43.685081 14924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:43.685292 14924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:43.692977 14924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:22:43.693377 14924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:43.693784 14924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:43.693938 14924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:22:43.694046 14924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:22:43.704593 14924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:46377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:43.705957 14924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:43.708185 14924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:43.721964 14930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:43.722074 14929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:43.722036 14932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:43.722653 14924 server_base.cc:1034] running on GCE node
I20260926 21:22:43.723567 14924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:43.725136 14924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:43.726352 14924 hybrid_clock.cc:648] HybridClock initialized: now 1790457763726296 us; error 41 us; skew 500 ppm
I20260926 21:22:43.726670 14924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:43 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457763, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:43.738103 14924 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:22:43.740049 14924 webserver.cc:460] Webserver started at http://127.14.142.62:37027/ using document root <none> and password file <none>
I20260926 21:22:43.741081 14924 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:43.741259 14924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:43.741633 14924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:43.745355 14924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "fd38050dae0444de9f9baa4182c6d9df"
format_stamp: "Formatted at 2026-09-26 21:22:43 on dist-test-slave-4d5t"
server_key: "46564df058c2e7232e2588dd5418974f"
server_key_iv: "b36cf4cc6348be71802ab21c8f0fd6b5"
server_key_version: "encryptionkey@0"
I20260926 21:22:43.746591 14924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "fd38050dae0444de9f9baa4182c6d9df"
format_stamp: "Formatted at 2026-09-26 21:22:43 on dist-test-slave-4d5t"
server_key: "46564df058c2e7232e2588dd5418974f"
server_key_iv: "b36cf4cc6348be71802ab21c8f0fd6b5"
server_key_version: "encryptionkey@0"
I20260926 21:22:43.754309 14924 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260926 21:22:43.758694 14939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:43.760227 14924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:22:43.760460 14924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "fd38050dae0444de9f9baa4182c6d9df"
format_stamp: "Formatted at 2026-09-26 21:22:43 on dist-test-slave-4d5t"
server_key: "46564df058c2e7232e2588dd5418974f"
server_key_iv: "b36cf4cc6348be71802ab21c8f0fd6b5"
server_key_version: "encryptionkey@0"
I20260926 21:22:43.760787 14924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:43.791353 14924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:43.796419 14924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:43.796799 14924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:43.818169 14924 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:46377
I20260926 21:22:43.818209 14990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:46377 every 8 connection(s)
I20260926 21:22:43.819772 14924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:22:43.824141 14991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:43.830406 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 14924
I20260926 21:22:43.830819 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:22:43.833614 14904 external_mini_cluster.cc:1428] Setting key 6c7c67da72e8cd09040fa2f77e32bd65
I20260926 21:22:43.838655 14991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df: Bootstrap starting.
I20260926 21:22:43.842433 14991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:43.843627 14991 log.cc:824] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:43.846854 14991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df: No bootstrap required, opened a new log
I20260926 21:22:43.857271 14991 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd38050dae0444de9f9baa4182c6d9df" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46377 } }
I20260926 21:22:43.857818 14991 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:43.857954 14991 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd38050dae0444de9f9baa4182c6d9df, State: Initialized, Role: FOLLOWER
I20260926 21:22:43.858709 14991 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd38050dae0444de9f9baa4182c6d9df" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46377 } }
I20260926 21:22:43.859043 14991 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:22:43.859220 14991 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:22:43.859423 14991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:43.863205 14991 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd38050dae0444de9f9baa4182c6d9df" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46377 } }
I20260926 21:22:43.863857 14991 leader_election.cc:302] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd38050dae0444de9f9baa4182c6d9df; no voters:
I20260926 21:22:43.864634 14991 leader_election.cc:288] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:22:43.864759 14996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:43.867723 14991 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:22:43.867936 14996 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [term 1 LEADER]: Becoming Leader. State: Replica: fd38050dae0444de9f9baa4182c6d9df, State: Running, Role: LEADER
Sep 26 21:22:43 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:43.868645 14996 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd38050dae0444de9f9baa4182c6d9df" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46377 } }
I20260926 21:22:43.873941 14998 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd38050dae0444de9f9baa4182c6d9df. Latest consensus state: current_term: 1 leader_uuid: "fd38050dae0444de9f9baa4182c6d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd38050dae0444de9f9baa4182c6d9df" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46377 } } }
I20260926 21:22:43.873603 14997 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd38050dae0444de9f9baa4182c6d9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd38050dae0444de9f9baa4182c6d9df" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46377 } } }
I20260926 21:22:43.874418 14998 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:43.874529 14997 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:43.877138 15003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:22:43.882632 15003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:22:43.889315 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:43.838826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.14.142.62:46377)
0926 21:22:43.839910 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:43.839943 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:43.841481 (+ 1538us) server_negotiation.cc:407] Connection header received
0926 21:22:43.852278 (+ 10797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:43.852310 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:43.852946 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:43.853441 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:43.856741 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:43.858899 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:43.861244 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:43.862632 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:43.862857 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:43.869922 (+ 7065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:43.869931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:43.869947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:43.869993 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:43.879793 (+ 9800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:43.881452 (+ 1659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:43.881466 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:43.881478 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:43.881781 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:43.882563 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:43.882576 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:43.882581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:43.883661 (+ 1080us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:43.884093 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:43.887476 (+ 3383us) server_negotiation.cc:299] Negotiation successful
0926 21:22:43.887930 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":497,"threads_started":1}
I20260926 21:22:43.897526 15003 catalog_manager.cc:1347] Generated new cluster ID: 7327359a4e69413eb4a49f82090f0ece
I20260926 21:22:43.897665 15003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:22:43.914042 15003 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:22:43.915835 15003 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:22:43.938797 15003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd38050dae0444de9f9baa4182c6d9df: Generated new TSK 0
I20260926 21:22:43.939765 15003 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:22:43.989697 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46377
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:44.342073 15017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:44.342394 15017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:44.342586 15017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:44.350235 15017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:44.350368 15017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:44.350569 15017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:22:44.361474 15017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46377
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:44.362922 15017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:44.364423 15017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:44.378710 15022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:44.378901 15025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:44.378917 15023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:44.380437 15017 server_base.cc:1034] running on GCE node
I20260926 21:22:44.381215 15017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:44.382278 15017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:44.383559 15017 hybrid_clock.cc:648] HybridClock initialized: now 1790457764383485 us; error 63 us; skew 500 ppm
I20260926 21:22:44.383976 15017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:44 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:44.388226 15017 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:22:44.390246 15017 webserver.cc:460] Webserver started at http://127.14.142.1:34031/ using document root <none> and password file <none>
I20260926 21:22:44.391078 15017 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:44.391268 15017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:44.391669 15017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:44.395309 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "3f47c2448cda4b8a85f158d56bf9b902"
format_stamp: "Formatted at 2026-09-26 21:22:44 on dist-test-slave-4d5t"
server_key: "b1f59589069800015a2acc9649623e2c"
server_key_iv: "5a8315df255eac6d34e15619b1cfb1b5"
server_key_version: "encryptionkey@0"
I20260926 21:22:44.396313 15017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "3f47c2448cda4b8a85f158d56bf9b902"
format_stamp: "Formatted at 2026-09-26 21:22:44 on dist-test-slave-4d5t"
server_key: "b1f59589069800015a2acc9649623e2c"
server_key_iv: "5a8315df255eac6d34e15619b1cfb1b5"
server_key_version: "encryptionkey@0"
I20260926 21:22:44.402678 15017 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 21:22:44.406586 15032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:44.408139 15017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:22:44.408360 15017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "3f47c2448cda4b8a85f158d56bf9b902"
format_stamp: "Formatted at 2026-09-26 21:22:44 on dist-test-slave-4d5t"
server_key: "b1f59589069800015a2acc9649623e2c"
server_key_iv: "5a8315df255eac6d34e15619b1cfb1b5"
server_key_version: "encryptionkey@0"
I20260926 21:22:44.408665 15017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:44.430610 15017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:44.435495 15017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:44.435866 15017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:44.437014 15017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:22:44.438813 15017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:44.438932 15017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:44.439071 15017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:44.439157 15017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:44.479043 15017 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:43431
I20260926 21:22:44.479161 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:43431 every 8 connection(s)
I20260926 21:22:44.480432 15017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:22:44.490224 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15017
I20260926 21:22:44.490502 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:22:44.491183 14904 external_mini_cluster.cc:1428] Setting key 9bdfbfa32cb22a2b7000e6bc63481406
Sep 26 21:22:44 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:44.503880 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:44.483267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:52397 (local address 127.14.142.62:46377)
0926 21:22:44.483768 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:44.483775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:44.484818 (+ 1043us) server_negotiation.cc:407] Connection header received
0926 21:22:44.486883 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:44.486889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:44.486993 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:44.487135 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:44.488960 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:44.489797 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:44.492421 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:44.493047 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:44.493226 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:44.497371 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:44.497377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:44.497381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:44.497420 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:44.500342 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:44.501282 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:44.501287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:44.501290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:44.501415 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:44.501900 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:44.501911 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:44.501913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:44.502209 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:44.502379 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:44.503192 (+ 813us) server_negotiation.cc:299] Negotiation successful
0926 21:22:44.503468 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":239,"threads_started":1}
I20260926 21:22:44.505052 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:44.483826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46377 (local address 127.14.142.1:52397)
0926 21:22:44.484644 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:44.484695 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:44.486550 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:44.487412 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:44.487422 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:44.488012 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:44.488764 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:44.488785 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:44.489980 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:44.489985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:44.492099 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:44.492135 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:44.493378 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:44.493391 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:44.493696 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:44.494482 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:44.494514 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:44.497067 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:44.500539 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:44.500576 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:44.500598 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:44.501111 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:44.501539 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:44.501545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:44.501549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:44.501767 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:44.502350 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:44.502360 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:44.502775 (+ 415us) client_negotiation.cc:769] Sending connection context
0926 21:22:44.503071 (+ 296us) client_negotiation.cc:240] Negotiation successful
0926 21:22:44.503708 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":280,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:22:44.506937 15145 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46377
I20260926 21:22:44.507274 15145 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:44.508178 15145 heartbeater.cc:511] Master 127.14.142.62:46377 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:22:44.511035 14956 ts_manager.cc:194] Registered new tserver with Master: 3f47c2448cda4b8a85f158d56bf9b902 (127.14.142.1:43431)
I20260926 21:22:44.512897 14956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:52397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:22:44.536661 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46377
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:44.891469 15151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:44.891801 15151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:44.891906 15151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:44.899595 15151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:44.899735 15151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:44.899855 15151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:22:44.909981 15151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46377
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:44.911314 15151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:44.912613 15151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:44.927501 15157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:44.927696 15159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:44.927532 15156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:44.928838 15151 server_base.cc:1034] running on GCE node
I20260926 21:22:44.929463 15151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:44.930450 15151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:44.931695 15151 hybrid_clock.cc:648] HybridClock initialized: now 1790457764931633 us; error 43 us; skew 500 ppm
I20260926 21:22:44.932068 15151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:44 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:44.935979 15151 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:22:44.938591 15151 webserver.cc:460] Webserver started at http://127.14.142.2:45509/ using document root <none> and password file <none>
I20260926 21:22:44.940097 15151 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:44.940267 15151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:44.940635 15151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:44.944005 15151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "1b9236f6328d4e13a4610c719a4553ac"
format_stamp: "Formatted at 2026-09-26 21:22:44 on dist-test-slave-4d5t"
server_key: "472689d5d43cee365e438b0b9c535fdc"
server_key_iv: "17fb658b927a5bb68fcce8d54a25f1d5"
server_key_version: "encryptionkey@0"
I20260926 21:22:44.944911 15151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "1b9236f6328d4e13a4610c719a4553ac"
format_stamp: "Formatted at 2026-09-26 21:22:44 on dist-test-slave-4d5t"
server_key: "472689d5d43cee365e438b0b9c535fdc"
server_key_iv: "17fb658b927a5bb68fcce8d54a25f1d5"
server_key_version: "encryptionkey@0"
I20260926 21:22:44.951238 15151 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:22:44.955211 15166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:44.956969 15151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 21:22:44.957136 15151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "1b9236f6328d4e13a4610c719a4553ac"
format_stamp: "Formatted at 2026-09-26 21:22:44 on dist-test-slave-4d5t"
server_key: "472689d5d43cee365e438b0b9c535fdc"
server_key_iv: "17fb658b927a5bb68fcce8d54a25f1d5"
server_key_version: "encryptionkey@0"
I20260926 21:22:44.957329 15151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:44.972421 15151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:44.977494 15151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:44.977777 15151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:44.978773 15151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:22:44.980180 15151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:44.980257 15151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:44.980341 15151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:44.980389 15151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:45.019731 15151 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33631
I20260926 21:22:45.019862 15278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33631 every 8 connection(s)
I20260926 21:22:45.021128 15151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:22:45.025653 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15151
I20260926 21:22:45.025882 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:22:45.026304 14904 external_mini_cluster.cc:1428] Setting key 6d0ca3fffe16c41c7469a121b67975f6
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457764, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:45.044065 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.023794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:36455 (local address 127.14.142.62:46377)
0926 21:22:45.024247 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.024258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.025292 (+ 1034us) server_negotiation.cc:407] Connection header received
0926 21:22:45.027297 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.027303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.027394 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.027528 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.029157 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.029867 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.031451 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.031898 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.032034 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.036534 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.036539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.036543 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.036654 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.039544 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.040715 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.040719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.040722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.040916 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.041513 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.041516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.041518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.041814 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.041929 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.043222 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.043637 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":235,"threads_started":1}
I20260926 21:22:45.045267 15282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.024344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46377 (local address 127.14.142.2:36455)
0926 21:22:45.025090 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:45.025142 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:45.027007 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:45.027770 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:45.027778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:45.028320 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:45.028975 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:45.028993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.030023 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.030034 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:45.031302 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:45.031314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.032147 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.032155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:45.032386 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.033333 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:45.033359 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:45.036193 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:45.039748 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:45.039760 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:45.039783 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:45.040513 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:45.041064 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:45.041069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:45.041073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:45.041369 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:45.041963 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:45.041975 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:45.042543 (+ 568us) client_negotiation.cc:769] Sending connection context
0926 21:22:45.042903 (+ 360us) client_negotiation.cc:240] Negotiation successful
0926 21:22:45.043578 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":316,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:22:45.047104 15279 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46377
I20260926 21:22:45.047534 15279 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
I20260926 21:22:45.048637 15279 heartbeater.cc:511] Master 127.14.142.62:46377 requested a full tablet report, sending...
I20260926 21:22:45.050441 14956 ts_manager.cc:194] Registered new tserver with Master: 1b9236f6328d4e13a4610c719a4553ac (127.14.142.2:33631)
I20260926 21:22:45.051792 14956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:36455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:22:45.077631 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46377
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:45.432319 15285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:45.432659 15285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:45.432840 15285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:45.440652 15285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:45.440784 15285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:45.441016 15285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:22:45.451884 15285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46377
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:45.453306 15285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:45.454706 15285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:45.470733 15290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:45.470780 15293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:45.471120 15291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:45.472211 15285 server_base.cc:1034] running on GCE node
I20260926 21:22:45.473201 15285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:45.474298 15285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:45.475811 15285 hybrid_clock.cc:648] HybridClock initialized: now 1790457765475716 us; error 69 us; skew 500 ppm
I20260926 21:22:45.476204 15285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:45.480067 15285 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:22:45.481714 15285 webserver.cc:460] Webserver started at http://127.14.142.3:37867/ using document root <none> and password file <none>
I20260926 21:22:45.482410 15285 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:45.482560 15285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:45.482962 15285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:45.486061 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "2a43b55babee4ac2a83cc2724259ebde"
format_stamp: "Formatted at 2026-09-26 21:22:45 on dist-test-slave-4d5t"
server_key: "e30c433c13e3a6de55f13f79124b1d64"
server_key_iv: "5da32b5a41435bded437d48be2e63996"
server_key_version: "encryptionkey@0"
I20260926 21:22:45.486933 15285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "2a43b55babee4ac2a83cc2724259ebde"
format_stamp: "Formatted at 2026-09-26 21:22:45 on dist-test-slave-4d5t"
server_key: "e30c433c13e3a6de55f13f79124b1d64"
server_key_iv: "5da32b5a41435bded437d48be2e63996"
server_key_version: "encryptionkey@0"
I20260926 21:22:45.492365 15285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 21:22:45.496163 15300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:45.497613 15285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:22:45.497831 15285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "2a43b55babee4ac2a83cc2724259ebde"
format_stamp: "Formatted at 2026-09-26 21:22:45 on dist-test-slave-4d5t"
server_key: "e30c433c13e3a6de55f13f79124b1d64"
server_key_iv: "5da32b5a41435bded437d48be2e63996"
server_key_version: "encryptionkey@0"
I20260926 21:22:45.498076 15285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:45.516511 15145 heartbeater.cc:503] Master 127.14.142.62:46377 was elected leader, sending a full tablet report...
I20260926 21:22:45.518275 15285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:45.522665 15285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:45.522990 15285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:45.524042 15285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:22:45.525632 15285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:45.525746 15285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:45.525918 15285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:45.526000 15285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:45.563633 15285 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:40543
I20260926 21:22:45.563728 15412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:40543 every 8 connection(s)
I20260926 21:22:45.565081 15285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:22:45.567696 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15285
I20260926 21:22:45.567885 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:22:45.568480 14904 external_mini_cluster.cc:1428] Setting key c926691639c98cf47fdb15533861374e
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:45.588928 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.567814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:50217 (local address 127.14.142.62:46377)
0926 21:22:45.568373 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.568379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.569683 (+ 1304us) server_negotiation.cc:407] Connection header received
0926 21:22:45.572263 (+ 2580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.572269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.572374 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.572499 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.574127 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.574993 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.576713 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.577144 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.577302 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.581825 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.581830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.581834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.581866 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.585447 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.586381 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.586387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.586392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.586563 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.587170 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.587174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.587177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.587473 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.587609 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.588378 (+ 769us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.588643 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":249,"threads_started":1}
I20260926 21:22:45.589907 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.568401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46377 (local address 127.14.142.3:50217)
0926 21:22:45.569488 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:45.569545 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:45.571945 (+ 2400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:45.572789 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:45.572798 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:45.573354 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:45.573951 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:45.573968 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.575161 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.575175 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:45.576510 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:45.576521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.577402 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.577411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:45.577693 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.578552 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:45.578576 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:45.581577 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:45.585602 (+ 4025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:45.585612 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:45.585630 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:45.586183 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:45.586707 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:45.586711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:45.586714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:45.587023 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:45.587602 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:45.587608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:45.588016 (+ 408us) client_negotiation.cc:769] Sending connection context
0926 21:22:45.588270 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 21:22:45.588781 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":381,"threads_started":1}
I20260926 21:22:45.591897 15413 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46377
I20260926 21:22:45.592255 15413 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:45.593113 15413 heartbeater.cc:511] Master 127.14.142.62:46377 requested a full tablet report, sending...
I20260926 21:22:45.594826 14956 ts_manager.cc:194] Registered new tserver with Master: 2a43b55babee4ac2a83cc2724259ebde (127.14.142.3:40543)
I20260926 21:22:45.595621 14956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:50217
I20260926 21:22:45.599227 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:22:45.646880 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.629074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.14.142.1:43431)
0926 21:22:45.629749 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.629762 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.629861 (+ 99us) server_negotiation.cc:407] Connection header received
0926 21:22:45.630087 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.630095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.630324 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.630507 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.632015 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.633129 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.634459 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.635046 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.635206 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.639635 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.639649 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.639668 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.639823 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.643734 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.644659 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.644667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.644669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.644818 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.645248 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.645251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.645253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.645614 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.645860 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.646271 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.646532 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":311,"threads_started":1}
W20260926 21:22:45.648243 15060 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54144
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:22:45.698170 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.680470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.14.142.1:43431)
0926 21:22:45.680790 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.680796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.680940 (+ 144us) server_negotiation.cc:407] Connection header received
0926 21:22:45.681113 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.681117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.681233 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.681332 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.683342 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.684056 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.686013 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.686474 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.686629 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.691764 (+ 5135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.691790 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.691794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.691842 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.694694 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.695827 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.695834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.695837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.696037 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.696734 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.696740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.696742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.697090 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.697305 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.697744 (+ 439us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.697935 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260926 21:22:45.699986 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15017
I20260926 21:22:45.714388 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.701126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.14.142.1:43431)
0926 21:22:45.701345 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.701356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.701556 (+ 200us) server_negotiation.cc:407] Connection header received
0926 21:22:45.701765 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.701770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.701832 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.701922 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.703940 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.704814 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.706145 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.706601 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.706710 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.709123 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.709127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.709131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.709163 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.711619 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.712289 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.712292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.712295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.712443 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.712857 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.712866 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.712868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.713122 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.713314 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.713802 (+ 488us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.714004 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:22:45.715135 15140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:45.800849 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15017
I20260926 21:22:45.818610 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15151
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:22:45.838316 15428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.819647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.14.142.2:33631)
0926 21:22:45.820204 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.820211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.820237 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:22:45.820298 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.820304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.820511 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.820726 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.822261 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.823265 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.825142 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.825713 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.825845 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.831316 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.831333 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.831349 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.831394 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.834892 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.835863 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.835875 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.835878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.836117 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.836728 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.836734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.836737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.837167 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.837435 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.837792 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.838015 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":296,"threads_started":1}
I20260926 21:22:45.839800 15274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:45.924249 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15151
I20260926 21:22:45.942039 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15285
Sep 26 21:22:45 dist-test-slave-4d5t krb5kdc[14911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:22:45.959440 15430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:45.943302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.14.142.3:40543)
0926 21:22:45.944084 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:45.944095 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:45.944129 (+ 34us) server_negotiation.cc:407] Connection header received
0926 21:22:45.944201 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:45.944208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:45.944415 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:45.944621 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:45.945846 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.946892 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.948190 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:45.948974 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:45.949085 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:45.953423 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:45.953437 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:45.953454 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:45.953496 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:45.956288 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.957279 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.957287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.957289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.957426 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:45.957852 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:45.957855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:45.957858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:45.958217 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:45.958453 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:45.958801 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 21:22:45.959009 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":401,"threads_started":1}
I20260926 21:22:45.960831 15408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:46.048908 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15285
I20260926 21:22:46.065654 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 14924
I20260926 21:22:46.067008 14986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:46.172737 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 14924
2026-09-26T21:22:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4176 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:22:46 dist-test-slave-4d5t krb5kdc[15435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:22:46 dist-test-slave-4d5t krb5kdc[15435](info): set up 2 sockets
Sep 26 21:22:46 dist-test-slave-4d5t krb5kdc[15435](info): commencing operation
krb5kdc: starting...
W20260926 21:22:47.289201 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.006s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:22:47 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:22:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:22:47.484241 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44355
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:47.836076 15448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:47.836433 15448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:47.836681 15448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:47.845563 15448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:22:47.845746 15448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:47.845885 15448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:47.845971 15448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:22:47.846045 15448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:22:47.857221 15448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:47.858701 15448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:47.860049 15448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:47.873320 15453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:47.873368 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:47.873409 15454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:47.874577 15448 server_base.cc:1034] running on GCE node
I20260926 21:22:47.875774 15448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:47.877844 15448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:47.879175 15448 hybrid_clock.cc:648] HybridClock initialized: now 1790457767879119 us; error 47 us; skew 500 ppm
I20260926 21:22:47.879645 15448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:47 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457767, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:47.883474 15448 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:22:47.885424 15448 webserver.cc:460] Webserver started at http://127.14.142.62:34079/ using document root <none> and password file <none>
I20260926 21:22:47.886193 15448 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:47.886353 15448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:47.886686 15448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:47.890475 15448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "5e909dde92424de39f8d01d65a8ae73a"
format_stamp: "Formatted at 2026-09-26 21:22:47 on dist-test-slave-4d5t"
server_key: "a8d49c68f3209bc3604bbe15d54d5c87"
server_key_iv: "11258231cb083c0f3d2df3d492f82eae"
server_key_version: "encryptionkey@0"
I20260926 21:22:47.891484 15448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "5e909dde92424de39f8d01d65a8ae73a"
format_stamp: "Formatted at 2026-09-26 21:22:47 on dist-test-slave-4d5t"
server_key: "a8d49c68f3209bc3604bbe15d54d5c87"
server_key_iv: "11258231cb083c0f3d2df3d492f82eae"
server_key_version: "encryptionkey@0"
I20260926 21:22:47.897089 15448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:22:47.901345 15463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:47.903368 15448 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:22:47.903594 15448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "5e909dde92424de39f8d01d65a8ae73a"
format_stamp: "Formatted at 2026-09-26 21:22:47 on dist-test-slave-4d5t"
server_key: "a8d49c68f3209bc3604bbe15d54d5c87"
server_key_iv: "11258231cb083c0f3d2df3d492f82eae"
server_key_version: "encryptionkey@0"
I20260926 21:22:47.903820 15448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:47.919428 15448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:47.923925 15448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:47.924228 15448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:47.945079 15448 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:44355
I20260926 21:22:47.945099 15514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:44355 every 8 connection(s)
I20260926 21:22:47.946488 15448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:22:47.951006 15515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:47.953886 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15448
I20260926 21:22:47.954169 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:22:47.954830 14904 external_mini_cluster.cc:1428] Setting key 82feb642d90ab1e94a61943fff6776ad
I20260926 21:22:47.964268 15515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a: Bootstrap starting.
I20260926 21:22:47.968604 15515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:47.970121 15515 log.cc:824] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a: Log is configured to *not* fsync() on all Append() calls
Sep 26 21:22:47 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:47.973996 15515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a: No bootstrap required, opened a new log
I20260926 21:22:47.981247 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:47.956945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.14.142.62:44355)
0926 21:22:47.957780 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:47.957794 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:47.957837 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:22:47.959694 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:47.959719 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:47.960147 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:47.960493 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:47.962571 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:47.963932 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:47.965709 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:47.966435 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:47.966582 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:47.972429 (+ 5847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:47.972437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:47.972453 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:47.972492 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:47.975833 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:47.976895 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:47.976904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:47.976912 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:47.977163 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:47.977943 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:47.977947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:47.977950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:47.978608 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:47.978916 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:47.979516 (+ 600us) server_negotiation.cc:299] Negotiation successful
0926 21:22:47.980034 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":396,"threads_started":1}
I20260926 21:22:47.985210 15515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e909dde92424de39f8d01d65a8ae73a" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44355 } }
I20260926 21:22:47.985762 15515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:47.985889 15515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e909dde92424de39f8d01d65a8ae73a, State: Initialized, Role: FOLLOWER
I20260926 21:22:47.986562 15515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e909dde92424de39f8d01d65a8ae73a" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44355 } }
I20260926 21:22:47.986830 15515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:22:47.986996 15515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:22:47.987191 15515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:47.990032 15515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e909dde92424de39f8d01d65a8ae73a" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44355 } }
I20260926 21:22:47.990670 15515 leader_election.cc:302] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e909dde92424de39f8d01d65a8ae73a; no voters:
I20260926 21:22:47.991353 15515 leader_election.cc:288] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:22:47.991499 15520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:47.992215 15520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [term 1 LEADER]: Becoming Leader. State: Replica: 5e909dde92424de39f8d01d65a8ae73a, State: Running, Role: LEADER
I20260926 21:22:47.992866 15520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e909dde92424de39f8d01d65a8ae73a" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44355 } }
I20260926 21:22:47.993875 15515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:22:47.996167 15521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e909dde92424de39f8d01d65a8ae73a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e909dde92424de39f8d01d65a8ae73a" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44355 } } }
I20260926 21:22:47.996629 15521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:47.997984 15522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e909dde92424de39f8d01d65a8ae73a. Latest consensus state: current_term: 1 leader_uuid: "5e909dde92424de39f8d01d65a8ae73a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e909dde92424de39f8d01d65a8ae73a" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44355 } } }
I20260926 21:22:48.000232 15522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:48.000913 15526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:22:48.007189 15526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:22:48.018926 15526 catalog_manager.cc:1347] Generated new cluster ID: 62719d85162a4f0b8dec46ff7de6d599
I20260926 21:22:48.019076 15526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:22:48.029045 15526 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:22:48.030670 15526 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:22:48.044996 15526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e909dde92424de39f8d01d65a8ae73a: Generated new TSK 0
I20260926 21:22:48.046027 15526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:22:48.109302 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44355
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:48.470618 15541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:48.470976 15541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:48.471146 15541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:48.478849 15541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:48.479033 15541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:48.479220 15541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:22:48.490023 15541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44355
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:48.491496 15541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:48.492993 15541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:48.507787 15547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:48.508153 15546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:48.508044 15549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:48.509047 15541 server_base.cc:1034] running on GCE node
I20260926 21:22:48.509838 15541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:48.511121 15541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:48.512713 15541 hybrid_clock.cc:648] HybridClock initialized: now 1790457768512471 us; error 73 us; skew 500 ppm
I20260926 21:22:48.513329 15541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:48 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457768, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:48.517171 15541 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:22:48.518801 15541 webserver.cc:460] Webserver started at http://127.14.142.1:35077/ using document root <none> and password file <none>
I20260926 21:22:48.520004 15541 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:48.520316 15541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:48.520772 15541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:48.524155 15541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "cfebefc0bd9f41109d102e6c6128d097"
format_stamp: "Formatted at 2026-09-26 21:22:48 on dist-test-slave-4d5t"
server_key: "90ba375e89a1d0496c7c046f4c6d4404"
server_key_iv: "98d3bce074656c27572806f43dd2ddc3"
server_key_version: "encryptionkey@0"
I20260926 21:22:48.525050 15541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "cfebefc0bd9f41109d102e6c6128d097"
format_stamp: "Formatted at 2026-09-26 21:22:48 on dist-test-slave-4d5t"
server_key: "90ba375e89a1d0496c7c046f4c6d4404"
server_key_iv: "98d3bce074656c27572806f43dd2ddc3"
server_key_version: "encryptionkey@0"
I20260926 21:22:48.531324 15541 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:22:48.535593 15556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:48.538028 15541 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:22:48.538240 15541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "cfebefc0bd9f41109d102e6c6128d097"
format_stamp: "Formatted at 2026-09-26 21:22:48 on dist-test-slave-4d5t"
server_key: "90ba375e89a1d0496c7c046f4c6d4404"
server_key_iv: "98d3bce074656c27572806f43dd2ddc3"
server_key_version: "encryptionkey@0"
I20260926 21:22:48.538491 15541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:48.556824 15541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:48.562489 15541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:48.562866 15541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:48.564060 15541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:22:48.565668 15541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:48.565778 15541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:48.565941 15541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:48.566016 15541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:48.603727 15541 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46173
I20260926 21:22:48.603761 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46173 every 8 connection(s)
I20260926 21:22:48.605063 15541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:22:48.612910 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15541
I20260926 21:22:48.613155 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:22:48.613763 14904 external_mini_cluster.cc:1428] Setting key ba901d74a38bfa6346562e4566476e2e
Sep 26 21:22:48 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457768, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:48.625972 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:48.607806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:34433 (local address 127.14.142.62:44355)
0926 21:22:48.608354 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:48.608359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:48.609406 (+ 1047us) server_negotiation.cc:407] Connection header received
0926 21:22:48.611265 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:48.611270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:48.611368 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:48.611490 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:48.613070 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:48.613839 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:48.615625 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:48.616025 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:48.616157 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:48.619927 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:48.619932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:48.619935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:48.619967 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:48.622799 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:48.623586 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:48.623596 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:48.623599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:48.623721 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:48.624178 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:48.624183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:48.624186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:48.624466 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:48.624651 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:48.625359 (+ 708us) server_negotiation.cc:299] Negotiation successful
0926 21:22:48.625589 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":323,"threads_started":1}
I20260926 21:22:48.626960 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:48.608467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44355 (local address 127.14.142.1:34433)
0926 21:22:48.609242 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:48.609279 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:48.610987 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:48.611716 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:48.611724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:48.612230 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:48.612905 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:48.612922 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:48.614012 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:48.614016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:48.615477 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:48.615499 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:48.616250 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:48.616259 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:48.616506 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:48.617250 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:48.617272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:48.619683 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:48.622961 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:48.622971 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:48.622988 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:48.623450 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:48.623833 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:48.623838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:48.623842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:48.624048 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:48.624612 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:48.624620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:48.624977 (+ 357us) client_negotiation.cc:769] Sending connection context
0926 21:22:48.625248 (+ 271us) client_negotiation.cc:240] Negotiation successful
0926 21:22:48.625795 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":288,"threads_started":1}
I20260926 21:22:48.628645 15669 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44355
I20260926 21:22:48.628953 15669 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:22:48.629730 15669 heartbeater.cc:511] Master 127.14.142.62:44355 requested a full tablet report, sending...
I20260926 21:22:48.632027 15480 ts_manager.cc:194] Registered new tserver with Master: cfebefc0bd9f41109d102e6c6128d097 (127.14.142.1:46173)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:22:48.633832 15480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:34433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:22:48.655764 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44355
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:49.011497 15675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:49.011793 15675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:49.011955 15675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:49.019680 15675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:49.019806 15675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:49.019985 15675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:22:49.030016 15675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44355
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:49.031414 15675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:49.032817 15675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:49.047436 15681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:49.047539 15683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:49.047519 15680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:49.048460 15675 server_base.cc:1034] running on GCE node
I20260926 21:22:49.049175 15675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:49.050343 15675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:49.051580 15675 hybrid_clock.cc:648] HybridClock initialized: now 1790457769051526 us; error 72 us; skew 500 ppm
I20260926 21:22:49.051966 15675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:49 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457769, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:49.062271 15675 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:22:49.063982 15675 webserver.cc:460] Webserver started at http://127.14.142.2:39651/ using document root <none> and password file <none>
I20260926 21:22:49.064836 15675 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:49.064985 15675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:49.065323 15675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:49.068344 15675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "97b14e8d7c3f428f8fe77ff94edbb82f"
format_stamp: "Formatted at 2026-09-26 21:22:49 on dist-test-slave-4d5t"
server_key: "004d4080aa0cb4aee4b095ab4df02bf7"
server_key_iv: "db6be63a27c4b798460f07742035328b"
server_key_version: "encryptionkey@0"
I20260926 21:22:49.069231 15675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "97b14e8d7c3f428f8fe77ff94edbb82f"
format_stamp: "Formatted at 2026-09-26 21:22:49 on dist-test-slave-4d5t"
server_key: "004d4080aa0cb4aee4b095ab4df02bf7"
server_key_iv: "db6be63a27c4b798460f07742035328b"
server_key_version: "encryptionkey@0"
I20260926 21:22:49.075191 15675 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:22:49.080163 15690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:49.081836 15675 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:22:49.082060 15675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "97b14e8d7c3f428f8fe77ff94edbb82f"
format_stamp: "Formatted at 2026-09-26 21:22:49 on dist-test-slave-4d5t"
server_key: "004d4080aa0cb4aee4b095ab4df02bf7"
server_key_iv: "db6be63a27c4b798460f07742035328b"
server_key_version: "encryptionkey@0"
I20260926 21:22:49.082319 15675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:49.110515 15675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:49.115012 15675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:49.115356 15675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:49.116415 15675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:22:49.117873 15675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:49.117980 15675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:49.118130 15675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:49.118202 15675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:49.157765 15675 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:40927
I20260926 21:22:49.157815 15802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:40927 every 8 connection(s)
I20260926 21:22:49.159082 15675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:22:49.166747 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15675
I20260926 21:22:49.166957 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:22:49.167502 14904 external_mini_cluster.cc:1428] Setting key 2a676aaa80269e84ce9abf8167da01dd
Sep 26 21:22:49 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457769, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:49.182883 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.161894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:39817 (local address 127.14.142.62:44355)
0926 21:22:49.162393 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.162400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.163478 (+ 1078us) server_negotiation.cc:407] Connection header received
0926 21:22:49.165632 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.165637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.165729 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.165834 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:49.168183 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.169182 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.171021 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.171477 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.171621 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.175594 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:49.175599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:49.175602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:49.175638 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:49.178462 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:49.179975 (+ 1513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:49.179981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:49.179984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:49.180270 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:49.180957 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:49.180965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:49.180969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:49.181284 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:49.181457 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.182222 (+ 765us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.182470 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":255,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:22:49.183972 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.162495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44355 (local address 127.14.142.2:39817)
0926 21:22:49.163264 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.163316 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.165320 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.166206 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.166214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.167053 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:49.167905 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.167931 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.169377 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.169381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.170821 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.170840 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.171752 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.171762 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.171990 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.172767 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:49.172792 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:49.175324 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:49.178632 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:49.178643 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:49.178665 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:49.179724 (+ 1059us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:49.180463 (+ 739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:49.180470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:49.180474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:49.180818 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:49.181428 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:49.181437 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:49.181839 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.182124 (+ 285us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.182729 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":311,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
I20260926 21:22:49.185650 15803 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44355
I20260926 21:22:49.186280 15803 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:49.187335 15803 heartbeater.cc:511] Master 127.14.142.62:44355 requested a full tablet report, sending...
I20260926 21:22:49.189280 15480 ts_manager.cc:194] Registered new tserver with Master: 97b14e8d7c3f428f8fe77ff94edbb82f (127.14.142.2:40927)
I20260926 21:22:49.190634 15480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:39817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:22:49.209589 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44355
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:49.539527 15809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:49.539847 15809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:49.539980 15809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:49.547648 15809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:49.547781 15809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:49.547967 15809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:22:49.558163 15809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44355
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:49.559576 15809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:49.561179 15809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:49.574914 15817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:49.574842 15814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:49.574882 15815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:49.576215 15809 server_base.cc:1034] running on GCE node
I20260926 21:22:49.576957 15809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:49.578572 15809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:49.579807 15809 hybrid_clock.cc:648] HybridClock initialized: now 1790457769579763 us; error 41 us; skew 500 ppm
I20260926 21:22:49.580188 15809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:49 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457769, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:49.584048 15809 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:22:49.585695 15809 webserver.cc:460] Webserver started at http://127.14.142.3:37309/ using document root <none> and password file <none>
I20260926 21:22:49.586457 15809 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:49.586611 15809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:49.586975 15809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:49.590149 15809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "b5400612e0004c5fa36a69e7eb5f1e9e"
format_stamp: "Formatted at 2026-09-26 21:22:49 on dist-test-slave-4d5t"
server_key: "5e1575410f43039b3304ede54f29c051"
server_key_iv: "caa9096c941956c3a519e9d421da90e5"
server_key_version: "encryptionkey@0"
I20260926 21:22:49.590968 15809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "b5400612e0004c5fa36a69e7eb5f1e9e"
format_stamp: "Formatted at 2026-09-26 21:22:49 on dist-test-slave-4d5t"
server_key: "5e1575410f43039b3304ede54f29c051"
server_key_iv: "caa9096c941956c3a519e9d421da90e5"
server_key_version: "encryptionkey@0"
I20260926 21:22:49.597144 15809 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 21:22:49.601051 15824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:49.602613 15809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:22:49.602809 15809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "b5400612e0004c5fa36a69e7eb5f1e9e"
format_stamp: "Formatted at 2026-09-26 21:22:49 on dist-test-slave-4d5t"
server_key: "5e1575410f43039b3304ede54f29c051"
server_key_iv: "caa9096c941956c3a519e9d421da90e5"
server_key_version: "encryptionkey@0"
I20260926 21:22:49.603466 15809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:49.634469 15809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:49.637144 15669 heartbeater.cc:503] Master 127.14.142.62:44355 was elected leader, sending a full tablet report...
I20260926 21:22:49.639221 15809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:49.639528 15809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:49.640795 15809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:22:49.642359 15809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:49.642433 15809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:49.642557 15809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:49.642606 15809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:49.681130 15809 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:43665
I20260926 21:22:49.681241 15936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:43665 every 8 connection(s)
I20260926 21:22:49.682534 15809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:22:49.688467 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 15809
I20260926 21:22:49.688721 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:22:49.689296 14904 external_mini_cluster.cc:1428] Setting key 743f5f6b256929b1192ec7cf6503ea7b
Sep 26 21:22:49 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457769, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:49.707846 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.686442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:33657 (local address 127.14.142.62:44355)
0926 21:22:49.687057 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.687063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.688214 (+ 1151us) server_negotiation.cc:407] Connection header received
0926 21:22:49.690977 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.690983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.691087 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.691221 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:49.693018 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.693862 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.695565 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.696015 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.696174 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.700009 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:49.700014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:49.700018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:49.700150 (+ 132us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:49.703538 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:49.704980 (+ 1442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:49.704985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:49.704988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:49.705236 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:49.705847 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:49.705874 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:49.705877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:49.706220 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:49.706364 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.707241 (+ 877us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.707553 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":247,"threads_started":1}
I20260926 21:22:49.709091 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.687065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44355 (local address 127.14.142.3:33657)
0926 21:22:49.687978 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.688048 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.690606 (+ 2558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.691479 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.691488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.692040 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:49.692836 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.692858 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.694032 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.694041 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.695387 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.695401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.696279 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.696287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.696541 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.697315 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:49.697339 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:49.699767 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:49.703836 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:49.703852 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:49.703881 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:49.704754 (+ 873us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:49.705391 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:49.705396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:49.705400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:49.705675 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:49.706355 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:49.706362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:49.706805 (+ 443us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.707088 (+ 283us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.707677 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":372,"threads_started":1}
I20260926 21:22:49.710569 15937 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44355
I20260926 21:22:49.710889 15937 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:49.711615 15937 heartbeater.cc:511] Master 127.14.142.62:44355 requested a full tablet report, sending...
I20260926 21:22:49.713459 15480 ts_manager.cc:194] Registered new tserver with Master: b5400612e0004c5fa36a69e7eb5f1e9e (127.14.142.3:43665)
I20260926 21:22:49.714368 15480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:33657
I20260926 21:22:49.718216 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:22:49.739565 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.725970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.14.142.62:44355)
0926 21:22:49.726154 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.726159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.726614 (+ 455us) server_negotiation.cc:407] Connection header received
0926 21:22:49.726740 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.726744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.726817 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.726927 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:49.729131 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.729907 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.731235 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.731630 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.731734 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.734162 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:49.734168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:49.734171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:49.734208 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:49.737029 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:49.737772 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:49.737776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:49.737778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:49.737901 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:49.738225 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:49.738228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:49.738230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:49.738574 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:49.738755 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.739111 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.739311 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 21:22:49.748471 15480 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:22:49.755558 15480 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:22:49.783025 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.770940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.14.142.3:43665)
0926 21:22:49.773121 (+ 2181us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.773133 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.774445 (+ 1312us) server_negotiation.cc:407] Connection header received
0926 21:22:49.774737 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.774754 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.774959 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.775118 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:49.775998 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.777556 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.780150 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.781559 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.781698 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.781954 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.782380 (+ 426us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.782560 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1982,"thread_start_us":1852,"threads_started":1}
I20260926 21:22:49.783201 15951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.770298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43665 (local address 127.0.0.1:46710)
0926 21:22:49.774329 (+ 4031us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.774352 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.774549 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.775262 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.775266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.775295 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:49.775699 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.775708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.777736 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.777744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.779974 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.779991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.781746 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.781752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.782106 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.782163 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.782321 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.782536 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3904,"mutex_wait_us":93,"thread_start_us":838,"threads_started":1}
I20260926 21:22:49.785002 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.769544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.14.142.1:46173)
0926 21:22:49.770277 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.770286 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.774074 (+ 3788us) server_negotiation.cc:407] Connection header received
0926 21:22:49.774402 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.774411 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.774627 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.774811 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:49.775887 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.777015 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.780876 (+ 3861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.782346 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.782488 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.782705 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.784376 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.784614 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":356,"threads_started":1}
I20260926 21:22:49.785035 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.768936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46173 (local address 127.0.0.1:42446)
0926 21:22:49.773938 (+ 5002us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.773963 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.774203 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.775115 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.775121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.775145 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:49.775676 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.775694 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.777196 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.777200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.780695 (+ 3495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.780711 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.783056 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.783081 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.784051 (+ 970us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.784095 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.784262 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.784441 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4871,"mutex_wait_us":235,"thread_start_us":5121,"threads_started":1}
I20260926 21:22:49.785961 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.768790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:40927 (local address 127.0.0.1:37664)
0926 21:22:49.771024 (+ 2234us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.771081 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.771279 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.774302 (+ 3023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.774312 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.774354 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:49.774806 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.774819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.777132 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.777136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.780695 (+ 3559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.780711 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.784782 (+ 4071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.784789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.785045 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.785094 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.785215 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.785378 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1961,"mutex_wait_us":235,"thread_start_us":4864,"threads_started":1}
I20260926 21:22:49.786844 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.768927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.14.142.2:40927)
0926 21:22:49.773600 (+ 4673us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.773610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.773643 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:22:49.773755 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.773764 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.774058 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.775033 (+ 975us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:49.775292 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.776973 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.782083 (+ 5110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.784104 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.784308 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.784536 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.786278 (+ 1742us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.786461 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4516,"thread_start_us":4393,"threads_started":1}
I20260926 21:22:49.788782 15872 tablet_service.cc:1467] Processing CreateTablet for tablet c5695c18f03f41559fdd45e8a574efec (DEFAULT_TABLE table=test-table [id=6833cffc9f9e4c778186f5c4ea049e31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:49.790537 15872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5695c18f03f41559fdd45e8a574efec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:49.790692 15604 tablet_service.cc:1467] Processing CreateTablet for tablet c5695c18f03f41559fdd45e8a574efec (DEFAULT_TABLE table=test-table [id=6833cffc9f9e4c778186f5c4ea049e31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:49.790889 15738 tablet_service.cc:1467] Processing CreateTablet for tablet c5695c18f03f41559fdd45e8a574efec (DEFAULT_TABLE table=test-table [id=6833cffc9f9e4c778186f5c4ea049e31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:49.791929 15604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5695c18f03f41559fdd45e8a574efec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:49.792243 15738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5695c18f03f41559fdd45e8a574efec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:49.809396 15956 tablet_bootstrap.cc:492] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: Bootstrap starting.
I20260926 21:22:49.812880 15957 tablet_bootstrap.cc:492] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: Bootstrap starting.
I20260926 21:22:49.813279 15956 tablet_bootstrap.cc:654] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:49.814633 15956 log.cc:824] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:49.815292 15958 tablet_bootstrap.cc:492] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: Bootstrap starting.
I20260926 21:22:49.816848 15957 tablet_bootstrap.cc:654] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:49.818185 15957 log.cc:824] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:49.819316 15958 tablet_bootstrap.cc:654] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:49.820081 15956 tablet_bootstrap.cc:492] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: No bootstrap required, opened a new log
I20260926 21:22:49.820377 15956 ts_tablet_manager.cc:1397] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260926 21:22:49.820704 15958 log.cc:824] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:49.821619 15957 tablet_bootstrap.cc:492] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: No bootstrap required, opened a new log
I20260926 21:22:49.821964 15957 ts_tablet_manager.cc:1397] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260926 21:22:49.827666 15958 tablet_bootstrap.cc:492] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: No bootstrap required, opened a new log
I20260926 21:22:49.828011 15958 ts_tablet_manager.cc:1397] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260926 21:22:49.829027 15956 raft_consensus.cc:348] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.829438 15956 raft_consensus.cc:374] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:49.829530 15956 raft_consensus.cc:729] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b14e8d7c3f428f8fe77ff94edbb82f, State: Initialized, Role: FOLLOWER
I20260926 21:22:49.830161 15956 consensus_queue.cc:260] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.831377 15803 heartbeater.cc:503] Master 127.14.142.62:44355 was elected leader, sending a full tablet report...
I20260926 21:22:49.833244 15956 ts_tablet_manager.cc:1428] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 21:22:49.838629 15958 raft_consensus.cc:348] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.838580 15957 raft_consensus.cc:348] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.839107 15958 raft_consensus.cc:374] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:49.839107 15957 raft_consensus.cc:374] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:49.839213 15958 raft_consensus.cc:729] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfebefc0bd9f41109d102e6c6128d097, State: Initialized, Role: FOLLOWER
I20260926 21:22:49.839213 15957 raft_consensus.cc:729] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5400612e0004c5fa36a69e7eb5f1e9e, State: Initialized, Role: FOLLOWER
I20260926 21:22:49.839883 15958 consensus_queue.cc:260] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.839888 15957 consensus_queue.cc:260] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.842310 15958 ts_tablet_manager.cc:1428] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: Time spent starting tablet: real 0.014s user 0.007s sys 0.008s
I20260926 21:22:49.843425 15937 heartbeater.cc:503] Master 127.14.142.62:44355 was elected leader, sending a full tablet report...
I20260926 21:22:49.844015 15957 ts_tablet_manager.cc:1428] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: Time spent starting tablet: real 0.022s user 0.012s sys 0.006s
W20260926 21:22:49.857748 15670 tablet.cc:2369] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:22:49.911331 15804 tablet.cc:2369] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:22:49.935154 15938 tablet.cc:2369] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:22:49.952737 15964 raft_consensus.cc:482] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:22:49.953114 15964 raft_consensus.cc:504] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.955384 15964 leader_election.cc:288] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b14e8d7c3f428f8fe77ff94edbb82f (127.14.142.2:40927), cfebefc0bd9f41109d102e6c6128d097 (127.14.142.1:46173)
I20260926 21:22:49.963994 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.955830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:49273 (local address 127.14.142.2:40927)
0926 21:22:49.956583 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.956591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.956787 (+ 196us) server_negotiation.cc:407] Connection header received
0926 21:22:49.957060 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.957065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.957148 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.957222 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:49.958007 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.959276 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.961909 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.963101 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.963276 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.963380 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.963625 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.963778 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260926 21:22:49.964645 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.955659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:40927 (local address 127.14.142.3:49273)
0926 21:22:49.956689 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.956707 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.956892 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.957400 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.957403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.957433 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:49.957814 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.957826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.959487 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.959491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.961731 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.961746 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.963314 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.963318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.963463 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.963489 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.963568 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.963906 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":280,"thread_start_us":1034,"threads_started":1}
I20260926 21:22:49.964942 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.956098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:36559 (local address 127.14.142.1:46173)
0926 21:22:49.956657 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:49.956663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:49.956689 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:22:49.956810 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:49.956815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:49.957716 (+ 901us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:49.958478 (+ 762us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:49.958711 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.960063 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.962246 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.963498 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.963677 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.963745 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:49.964493 (+ 748us) server_negotiation.cc:299] Negotiation successful
0926 21:22:49.964665 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 21:22:49.965006 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:49.955659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46173 (local address 127.14.142.3:36559)
0926 21:22:49.955968 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:49.955989 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:49.956131 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:49.957953 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:49.957958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:49.958009 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:49.958297 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.958308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.960241 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.960246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.962070 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:49.962080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:49.963688 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:49.963694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:49.964210 (+ 516us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:49.964270 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 21:22:49.964470 (+ 200us) client_negotiation.cc:240] Negotiation successful
0926 21:22:49.964710 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":1}
I20260926 21:22:49.966082 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5695c18f03f41559fdd45e8a574efec" candidate_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" is_pre_election: true
I20260926 21:22:49.966122 15624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5695c18f03f41559fdd45e8a574efec" candidate_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfebefc0bd9f41109d102e6c6128d097" is_pre_election: true
I20260926 21:22:49.966667 15758 raft_consensus.cc:2454] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5400612e0004c5fa36a69e7eb5f1e9e in term 0.
I20260926 21:22:49.966856 15624 raft_consensus.cc:2454] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5400612e0004c5fa36a69e7eb5f1e9e in term 0.
I20260926 21:22:49.967633 15825 leader_election.cc:302] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 97b14e8d7c3f428f8fe77ff94edbb82f, b5400612e0004c5fa36a69e7eb5f1e9e, cfebefc0bd9f41109d102e6c6128d097; no voters:
I20260926 21:22:49.968251 15964 raft_consensus.cc:2781] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:22:49.968520 15964 raft_consensus.cc:482] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:22:49.968674 15964 raft_consensus.cc:3037] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:49.971241 15964 raft_consensus.cc:504] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.971894 15964 leader_election.cc:288] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [CANDIDATE]: Term 1 election: Requested vote from peers 97b14e8d7c3f428f8fe77ff94edbb82f (127.14.142.2:40927), cfebefc0bd9f41109d102e6c6128d097 (127.14.142.1:46173)
I20260926 21:22:49.972373 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5695c18f03f41559fdd45e8a574efec" candidate_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f"
I20260926 21:22:49.972460 15624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5695c18f03f41559fdd45e8a574efec" candidate_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfebefc0bd9f41109d102e6c6128d097"
I20260926 21:22:49.972635 15758 raft_consensus.cc:3037] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:49.972761 15624 raft_consensus.cc:3037] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:49.975042 15758 raft_consensus.cc:2454] T c5695c18f03f41559fdd45e8a574efec P 97b14e8d7c3f428f8fe77ff94edbb82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5400612e0004c5fa36a69e7eb5f1e9e in term 1.
I20260926 21:22:49.975047 15624 raft_consensus.cc:2454] T c5695c18f03f41559fdd45e8a574efec P cfebefc0bd9f41109d102e6c6128d097 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5400612e0004c5fa36a69e7eb5f1e9e in term 1.
I20260926 21:22:49.975579 15826 leader_election.cc:302] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 97b14e8d7c3f428f8fe77ff94edbb82f, b5400612e0004c5fa36a69e7eb5f1e9e, cfebefc0bd9f41109d102e6c6128d097; no voters:
I20260926 21:22:49.975952 15964 raft_consensus.cc:2781] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:49.976435 15964 raft_consensus.cc:686] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [term 1 LEADER]: Becoming Leader. State: Replica: b5400612e0004c5fa36a69e7eb5f1e9e, State: Running, Role: LEADER
I20260926 21:22:49.976940 15964 consensus_queue.cc:237] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } }
I20260926 21:22:49.982772 15479 catalog_manager.cc:5495] T c5695c18f03f41559fdd45e8a574efec P b5400612e0004c5fa36a69e7eb5f1e9e reported cstate change: term changed from 0 to 1, leader changed from <none> to b5400612e0004c5fa36a69e7eb5f1e9e (127.14.142.3). New cstate: current_term: 1 leader_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b14e8d7c3f428f8fe77ff94edbb82f" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 40927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfebefc0bd9f41109d102e6c6128d097" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5400612e0004c5fa36a69e7eb5f1e9e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43665 } health_report { overall_health: HEALTHY } } }
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:22:50.036440 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:50.018721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.14.142.1:46173)
0926 21:22:50.018967 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:50.018972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:50.019594 (+ 622us) server_negotiation.cc:407] Connection header received
0926 21:22:50.019856 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:50.019862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:50.019940 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:50.020078 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:50.021997 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.022810 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.024219 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.024784 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.024942 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:50.029646 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:50.029665 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:50.029683 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:50.029724 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:50.033513 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.034290 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.034299 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.034301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.034446 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.034819 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.034826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.034841 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.035200 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:50.035517 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:50.035816 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 21:22:50.036056 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457770, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457770, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:22:50.087594 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:50.071582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.14.142.1:46173)
0926 21:22:50.071805 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:50.071814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:50.071995 (+ 181us) server_negotiation.cc:407] Connection header received
0926 21:22:50.072223 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:50.072227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:50.072298 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:50.072404 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:50.073907 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.074656 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.075918 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.076617 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.076756 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:50.081591 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:50.081597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:50.081601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:50.081644 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:50.084506 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.085335 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.085340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.085342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.085479 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.086032 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.086041 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.086043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.086351 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:50.086633 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:50.087095 (+ 462us) server_negotiation.cc:299] Negotiation successful
0926 21:22:50.087305 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:22:50.135008 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:50.120290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.14.142.1:46173)
0926 21:22:50.120604 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:50.120613 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:50.121001 (+ 388us) server_negotiation.cc:407] Connection header received
0926 21:22:50.121227 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:50.121231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:50.121299 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:50.121406 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:50.122816 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.123525 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.124827 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.125261 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.125403 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:50.129633 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:50.129638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:50.129642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:50.129688 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:50.132214 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.132982 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.132987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.132991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.133124 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.133502 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.133507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.133510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.133839 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:50.134090 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:50.134401 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 21:22:50.134654 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260926 21:22:50.137444 15584 tablet_service.cc:2908] Rejecting scan request for tablet c5695c18f03f41559fdd45e8a574efec: Uninitialized: clean time has not yet been initialized
I20260926 21:22:50.141475 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15541
I20260926 21:22:50.157081 15953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:50.142889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.14.142.1:46173)
0926 21:22:50.143228 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:50.143236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:50.143533 (+ 297us) server_negotiation.cc:407] Connection header received
0926 21:22:50.143818 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:50.143824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:50.143998 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:50.144117 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:50.145526 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.146298 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.147634 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.148114 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.148279 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:50.150911 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:50.150917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:50.150921 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:50.150958 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:50.153911 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.154745 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.154752 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.154757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.154913 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.155307 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.155314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.155318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.155644 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:50.155978 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:50.156293 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 21:22:50.156597 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260926 21:22:50.157740 15664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:50.261597 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15541
I20260926 21:22:50.278995 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15675
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:22:50.297556 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:50.279964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.14.142.2:40927)
0926 21:22:50.280213 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:50.280220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:50.280247 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:22:50.280305 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:50.280309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:50.280374 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:50.280492 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:50.282010 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.282773 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.284260 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.284830 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.284965 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:50.290106 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:50.290122 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:50.290139 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:50.290182 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:50.293531 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.294537 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.294544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.294546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.294686 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.295395 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.295401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.295404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.295996 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:50.296418 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:50.296760 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 21:22:50.297176 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 21:22:50.298336 15798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:50.393224 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15675
I20260926 21:22:50.411257 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15809
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:22:50.429561 15981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:50.412049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.14.142.3:43665)
0926 21:22:50.412706 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:50.412710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:50.412728 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:22:50.412788 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:50.412792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:50.412856 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:50.413004 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:50.414373 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.415111 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.416734 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:50.417153 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:50.417291 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:50.422563 (+ 5272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:50.422581 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:50.422599 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:50.422641 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:50.426069 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.427177 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.427185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.427188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.427378 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:50.427878 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:50.427881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:50.427883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:50.428237 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:50.428473 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:50.428950 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 21:22:50.429182 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":344,"threads_started":1}
I20260926 21:22:50.430325 15932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:50.524576 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15809
I20260926 21:22:50.541617 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 15448
I20260926 21:22:50.542966 15510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:50.655359 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 15448
2026-09-26T21:22:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4484 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15987](info): set up 2 sockets
Sep 26 21:22:50 dist-test-slave-4d5t krb5kdc[15987](info): commencing operation
krb5kdc: starting...
W20260926 21:22:51.797400 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:22:51 dist-test-slave-4d5t krb5kdc[15987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:22:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:22:52.001403 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38717
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:38717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:52.356696 16000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:52.357088 16000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:52.357259 16000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:52.365199 16000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:22:52.365387 16000 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 21:22:52.365485 16000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:52.365604 16000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:52.365690 16000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:22:52.365795 16000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:22:52.376258 16000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:38717
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:52.377727 16000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:52.379052 16000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:52.391252 16005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:52.391389 16008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:52.391407 16006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:52.392395 16000 server_base.cc:1034] running on GCE node
I20260926 21:22:52.393195 16000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:52.394671 16000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:52.395929 16000 hybrid_clock.cc:648] HybridClock initialized: now 1790457772395844 us; error 60 us; skew 500 ppm
I20260926 21:22:52.396288 16000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:52 dist-test-slave-4d5t krb5kdc[15987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457772, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:52.399724 16000 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:22:52.401520 16000 webserver.cc:460] Webserver started at http://127.14.142.62:43733/ using document root <none> and password file <none>
I20260926 21:22:52.402338 16000 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:52.402470 16000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:52.402810 16000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:52.405887 16000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "70a25af7a6d64ffe9c76d853767fae5c"
format_stamp: "Formatted at 2026-09-26 21:22:52 on dist-test-slave-4d5t"
server_key: "bc21a563a2bb7456368e7ce5dd9fe8db"
server_key_iv: "9c92414c6e115e390e68c11c1aae90dd"
server_key_version: "encryptionkey@0"
I20260926 21:22:52.406685 16000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "70a25af7a6d64ffe9c76d853767fae5c"
format_stamp: "Formatted at 2026-09-26 21:22:52 on dist-test-slave-4d5t"
server_key: "bc21a563a2bb7456368e7ce5dd9fe8db"
server_key_iv: "9c92414c6e115e390e68c11c1aae90dd"
server_key_version: "encryptionkey@0"
I20260926 21:22:52.411922 16000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 21:22:52.415555 16015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:52.416954 16000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:22:52.417150 16000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "70a25af7a6d64ffe9c76d853767fae5c"
format_stamp: "Formatted at 2026-09-26 21:22:52 on dist-test-slave-4d5t"
server_key: "bc21a563a2bb7456368e7ce5dd9fe8db"
server_key_iv: "9c92414c6e115e390e68c11c1aae90dd"
server_key_version: "encryptionkey@0"
I20260926 21:22:52.417399 16000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:52.431793 16000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:52.436827 16000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:52.437142 16000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:52.460440 16000 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:38717
I20260926 21:22:52.460480 16076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:38717 every 8 connection(s)
I20260926 21:22:52.461861 16000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:22:52.463723 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16000
I20260926 21:22:52.463985 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:22:52.464833 14904 external_mini_cluster.cc:1428] Setting key 960b8f4988915e7c1ca456cff7b5c2f1
I20260926 21:22:52.466523 16077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:52.480752 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: Bootstrap starting.
Sep 26 21:22:52 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:52.484350 16077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:52.485483 16077 log.cc:824] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:52.488348 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: No bootstrap required, opened a new log
I20260926 21:22:52.490857 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:52.467399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.14.142.62:38717)
0926 21:22:52.468482 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:52.468496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:52.468539 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:22:52.471304 (+ 2765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:52.471330 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:52.471740 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:52.472104 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:52.473881 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:52.475168 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:52.477034 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:52.478120 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:52.478328 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:52.483284 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:52.483293 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:52.483306 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:52.483353 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:52.486751 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:52.487503 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:52.487509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:52.487515 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:52.487691 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:52.488073 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:52.488077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:52.488079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:52.488719 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:52.489063 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:52.489422 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 21:22:52.489856 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":594,"threads_started":1}
I20260926 21:22:52.496966 16077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a25af7a6d64ffe9c76d853767fae5c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38717 } }
I20260926 21:22:52.497408 16077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:52.497555 16077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a25af7a6d64ffe9c76d853767fae5c, State: Initialized, Role: FOLLOWER
I20260926 21:22:52.498162 16077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a25af7a6d64ffe9c76d853767fae5c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38717 } }
I20260926 21:22:52.498385 16077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:22:52.498551 16077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:22:52.498759 16077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:52.501667 16077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a25af7a6d64ffe9c76d853767fae5c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38717 } }
I20260926 21:22:52.502230 16077 leader_election.cc:302] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70a25af7a6d64ffe9c76d853767fae5c; no voters:
I20260926 21:22:52.502847 16077 leader_election.cc:288] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:22:52.503012 16082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:52.503811 16082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [term 1 LEADER]: Becoming Leader. State: Replica: 70a25af7a6d64ffe9c76d853767fae5c, State: Running, Role: LEADER
I20260926 21:22:52.504364 16082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a25af7a6d64ffe9c76d853767fae5c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38717 } }
I20260926 21:22:52.505285 16077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:22:52.507092 16083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70a25af7a6d64ffe9c76d853767fae5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a25af7a6d64ffe9c76d853767fae5c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38717 } } }
I20260926 21:22:52.507546 16083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:52.509398 16084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a25af7a6d64ffe9c76d853767fae5c. Latest consensus state: current_term: 1 leader_uuid: "70a25af7a6d64ffe9c76d853767fae5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a25af7a6d64ffe9c76d853767fae5c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38717 } } }
I20260926 21:22:52.509915 16084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:52.512063 16090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:22:52.517565 16090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:22:52.528658 16090 catalog_manager.cc:1347] Generated new cluster ID: 29196fb6b3744855b66866c714e00c9b
I20260926 21:22:52.528796 16090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:22:52.550915 16090 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:22:52.552891 16090 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:22:52.560710 16090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: Generated new TSK 0
I20260926 21:22:52.561589 16090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:22:52.620522 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38717
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:52.986549 16104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:52.986860 16104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:52.987035 16104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:52.995615 16104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:52.995752 16104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:52.995888 16104 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:22:52.996009 16104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:22:53.006553 16104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:38717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:53.007896 16104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:53.009737 16104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:53.023603 16110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:53.023576 16112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:53.023554 16109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:53.024433 16104 server_base.cc:1034] running on GCE node
I20260926 21:22:53.025110 16104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:53.026170 16104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:53.027420 16104 hybrid_clock.cc:648] HybridClock initialized: now 1790457773027265 us; error 100 us; skew 500 ppm
I20260926 21:22:53.027743 16104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:53 dist-test-slave-4d5t krb5kdc[15987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457773, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:53.031577 16104 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:22:53.033373 16104 webserver.cc:460] Webserver started at http://127.14.142.1:35735/ using document root <none> and password file <none>
I20260926 21:22:53.034124 16104 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:53.034216 16104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:53.034574 16104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:53.037967 16104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "38a5c24d80234837ac3ae14ddbcd1ee8"
format_stamp: "Formatted at 2026-09-26 21:22:53 on dist-test-slave-4d5t"
server_key: "8ae784f1174f4a00491aa9eeca0b7494"
server_key_iv: "2139b044a6a4ad1adf20f02b0cb871ab"
server_key_version: "encryptionkey@0"
I20260926 21:22:53.038950 16104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "38a5c24d80234837ac3ae14ddbcd1ee8"
format_stamp: "Formatted at 2026-09-26 21:22:53 on dist-test-slave-4d5t"
server_key: "8ae784f1174f4a00491aa9eeca0b7494"
server_key_iv: "2139b044a6a4ad1adf20f02b0cb871ab"
server_key_version: "encryptionkey@0"
I20260926 21:22:53.044993 16104 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:22:53.049324 16119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:53.051447 16104 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:22:53.051676 16104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "38a5c24d80234837ac3ae14ddbcd1ee8"
format_stamp: "Formatted at 2026-09-26 21:22:53 on dist-test-slave-4d5t"
server_key: "8ae784f1174f4a00491aa9eeca0b7494"
server_key_iv: "2139b044a6a4ad1adf20f02b0cb871ab"
server_key_version: "encryptionkey@0"
I20260926 21:22:53.052019 16104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:53.088938 16104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:53.093499 16104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:53.093732 16104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:53.096532 16104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:53.096660 16104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260926 21:22:53.096834 16104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:53.096903 16104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:22:53 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457773, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:53.122143 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.096984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:35609 (local address 127.14.142.62:38717)
0926 21:22:53.097437 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:53.097445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:53.098669 (+ 1224us) server_negotiation.cc:407] Connection header received
0926 21:22:53.100954 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:53.100960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:53.101053 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:53.101179 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:53.103431 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.104746 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.106645 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.107430 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.107583 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.111998 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:53.112007 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:53.112014 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:53.112062 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:53.116700 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.117955 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.117959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.117961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.118124 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.118777 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.118785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.118790 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.119490 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:53.119765 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:53.121497 (+ 1732us) server_negotiation.cc:299] Negotiation successful
0926 21:22:53.121768 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":223,"threads_started":1}
I20260926 21:22:53.123718 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.097735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.14.142.1:35609)
0926 21:22:53.098516 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:53.098556 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:53.100625 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:53.101428 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:53.101441 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:53.102237 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:53.103151 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.103180 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.104925 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.104929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.106475 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.106493 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.107696 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.107702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.107889 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.108720 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:53.108742 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:53.111737 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:53.116892 (+ 5155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.116911 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.116937 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.117752 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.118294 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.118305 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.118311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.118627 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.119888 (+ 1261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:53.119902 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:53.120921 (+ 1019us) client_negotiation.cc:769] Sending connection context
0926 21:22:53.121370 (+ 449us) client_negotiation.cc:240] Negotiation successful
0926 21:22:53.121995 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":352,"threads_started":1}
I20260926 21:22:53.149346 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.129948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:33665 (local address 127.14.142.62:38717)
0926 21:22:53.130202 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:53.130211 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:53.133432 (+ 3221us) server_negotiation.cc:407] Connection header received
0926 21:22:53.133685 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:53.133698 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:53.133771 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:53.133896 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:53.135355 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.136207 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.138407 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.139007 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.139144 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.141910 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:53.141920 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:53.141925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:53.141971 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:53.145638 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.146684 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.146693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.146697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.146877 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.147391 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.147396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.147399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.147831 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:53.147988 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:53.148719 (+ 731us) server_negotiation.cc:299] Negotiation successful
0926 21:22:53.148953 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:22:53.149514 16193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.129896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.14.142.1:33665)
0926 21:22:53.133254 (+ 3358us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:53.133285 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:53.133512 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:53.133939 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:53.133943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:53.134503 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:53.135116 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.135136 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.136410 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.136418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.138170 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.138183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.139395 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.139399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.139668 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.140462 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:53.140493 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:53.141619 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:53.145815 (+ 4196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.145823 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.145828 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.146501 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.147012 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.147020 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.147025 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.147239 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.148041 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:53.148045 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:53.148310 (+ 265us) client_negotiation.cc:769] Sending connection context
0926 21:22:53.148490 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 21:22:53.148954 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3124,"thread_start_us":2849,"threads_started":1}
I20260926 21:22:53.151640 16104 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46525
I20260926 21:22:53.151640 16239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46525 every 8 connection(s)
I20260926 21:22:53.153048 16104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:22:53.154363 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16104
I20260926 21:22:53.154592 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:22:53.155309 14904 external_mini_cluster.cc:1428] Setting key a0cdaedb3d65602a633083c4e0215ebe
I20260926 21:22:53.155664 16240 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38717
I20260926 21:22:53.155884 16240 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:53.156621 16240 heartbeater.cc:511] Master 127.14.142.62:38717 requested a full tablet report, sending...
I20260926 21:22:53.159397 16032 ts_manager.cc:194] Registered new tserver with Master: 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1:46525)
I20260926 21:22:53.161782 16032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:35609
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:22:53.201050 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38717
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:53.545706 16244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:53.546087 16244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:53.546268 16244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:53.554021 16244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:53.554208 16244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:53.554382 16244 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:22:53.554528 16244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:22:53.564822 16244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:38717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:53.566236 16244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:53.567621 16244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:53.581743 16249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:53.581754 16252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:53.581691 16250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:53.583498 16244 server_base.cc:1034] running on GCE node
I20260926 21:22:53.584158 16244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:53.585314 16244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:53.586558 16244 hybrid_clock.cc:648] HybridClock initialized: now 1790457773586491 us; error 48 us; skew 500 ppm
I20260926 21:22:53.586932 16244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:53 dist-test-slave-4d5t krb5kdc[15987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457773, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:53.590977 16244 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:22:53.592913 16244 webserver.cc:460] Webserver started at http://127.14.142.2:36235/ using document root <none> and password file <none>
I20260926 21:22:53.593770 16244 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:53.593896 16244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:53.594273 16244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:53.597595 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "e6125a01dbbc42649ea2ae5c05a8673e"
format_stamp: "Formatted at 2026-09-26 21:22:53 on dist-test-slave-4d5t"
server_key: "03f358bb02a18cfa335058690a345381"
server_key_iv: "8868057716d2588651a81ea701ff4d9e"
server_key_version: "encryptionkey@0"
I20260926 21:22:53.598474 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "e6125a01dbbc42649ea2ae5c05a8673e"
format_stamp: "Formatted at 2026-09-26 21:22:53 on dist-test-slave-4d5t"
server_key: "03f358bb02a18cfa335058690a345381"
server_key_iv: "8868057716d2588651a81ea701ff4d9e"
server_key_version: "encryptionkey@0"
I20260926 21:22:53.604532 16244 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 21:22:53.608827 16259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:53.610469 16244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 21:22:53.610651 16244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "e6125a01dbbc42649ea2ae5c05a8673e"
format_stamp: "Formatted at 2026-09-26 21:22:53 on dist-test-slave-4d5t"
server_key: "03f358bb02a18cfa335058690a345381"
server_key_iv: "8868057716d2588651a81ea701ff4d9e"
server_key_version: "encryptionkey@0"
I20260926 21:22:53.610877 16244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:53.634315 16244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:53.639218 16244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:53.639568 16244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:53.642786 16244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:53.643142 16244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:53.643483 16244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:53.643631 16244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:22:53 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457773, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:53.666905 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.643222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:55461 (local address 127.14.142.62:38717)
0926 21:22:53.643556 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:53.643565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:53.645180 (+ 1615us) server_negotiation.cc:407] Connection header received
0926 21:22:53.647464 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:53.647475 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:53.647618 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:53.647798 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:53.649651 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.650977 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.653925 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.654377 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.654521 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.659246 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:53.659251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:53.659255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:53.659293 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:53.662765 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.663877 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.663887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.663893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.664090 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.664593 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.664600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.664606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.665062 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:53.665276 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:53.666019 (+ 743us) server_negotiation.cc:299] Negotiation successful
0926 21:22:53.666364 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260926 21:22:53.667510 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.643679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.14.142.2:55461)
0926 21:22:53.644959 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:53.645016 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:53.647101 (+ 2085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:53.648059 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:53.648068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:53.648730 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:53.649358 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.649407 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.651197 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.651205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.653714 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.653741 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.654650 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.654658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.654879 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.655600 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:53.655629 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:53.658967 (+ 3338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:53.662925 (+ 3958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.662941 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.662963 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.663696 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.664241 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.664244 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.664247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.664445 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.665211 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:53.665218 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:53.665603 (+ 385us) client_negotiation.cc:769] Sending connection context
0926 21:22:53.665836 (+ 233us) client_negotiation.cc:240] Negotiation successful
0926 21:22:53.666258 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":555,"threads_started":1}
I20260926 21:22:53.696465 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.676884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:41505 (local address 127.14.142.62:38717)
0926 21:22:53.677119 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:53.677128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:53.678166 (+ 1038us) server_negotiation.cc:407] Connection header received
0926 21:22:53.678386 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:53.678394 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:53.678498 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:53.678607 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:53.681097 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.682412 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.685002 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.685766 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.685935 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.688502 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:53.688511 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:53.688519 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:53.688613 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:53.693059 (+ 4446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.694123 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.694127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.694129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.694275 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:53.694795 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:53.694807 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:53.694810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:53.695296 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:53.695457 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:53.695862 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 21:22:53.696099 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:22:53.696733 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:53.677117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.14.142.2:41505)
0926 21:22:53.678045 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:53.678062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:53.678241 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:53.678688 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:53.678697 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:53.680189 (+ 1492us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:53.680833 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.680854 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.682598 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.682606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.684798 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:53.684822 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:53.685980 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:53.685985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:53.686101 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:53.686864 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:53.686887 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:53.688181 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:53.693279 (+ 5098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.693290 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.693294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.693928 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.694458 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:53.694466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:53.694471 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:53.694662 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:53.695416 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:53.695421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:53.695607 (+ 186us) client_negotiation.cc:769] Sending connection context
0926 21:22:53.695755 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 21:22:53.696200 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":660,"threads_started":1}
I20260926 21:22:53.697919 16244 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:38555
I20260926 21:22:53.697943 16378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:38555 every 8 connection(s)
I20260926 21:22:53.699565 16244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:22:53.701390 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16244
I20260926 21:22:53.701617 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:22:53.702040 14904 external_mini_cluster.cc:1428] Setting key 29d97291288ba6d0197a7243201e79ab
I20260926 21:22:53.701826 16379 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38717
I20260926 21:22:53.703176 16379 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:53.703933 16379 heartbeater.cc:511] Master 127.14.142.62:38717 requested a full tablet report, sending...
I20260926 21:22:53.705478 16032 ts_manager.cc:194] Registered new tserver with Master: e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555)
I20260926 21:22:53.706530 16032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:55461
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:22:53.748476 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38717
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:54.098485 16383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:54.098855 16383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:54.099037 16383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:54.107854 16383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:54.108044 16383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:54.108218 16383 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:22:54.108397 16383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:22:54.120086 16383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:38717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:54.121697 16383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:54.123191 16383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:54.137763 16388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:54.138231 16391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:54.138273 16389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:54.138861 16383 server_base.cc:1034] running on GCE node
I20260926 21:22:54.139493 16383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:54.140507 16383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:54.141839 16383 hybrid_clock.cc:648] HybridClock initialized: now 1790457774141762 us; error 42 us; skew 500 ppm
I20260926 21:22:54.142198 16383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:54 dist-test-slave-4d5t krb5kdc[15987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:54.151973 16383 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:22:54.155520 16383 webserver.cc:460] Webserver started at http://127.14.142.3:41265/ using document root <none> and password file <none>
I20260926 21:22:54.156306 16383 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:54.156437 16383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:54.156881 16383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:54.160229 16383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "77ffdb46a34f4f7697acc91af7baef57"
format_stamp: "Formatted at 2026-09-26 21:22:54 on dist-test-slave-4d5t"
server_key: "8fbbd56ec2fb02b94ddcdb330083a698"
server_key_iv: "2fc4d1e2d700c5ed93791631cb9886f5"
server_key_version: "encryptionkey@0"
I20260926 21:22:54.161235 16383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "77ffdb46a34f4f7697acc91af7baef57"
format_stamp: "Formatted at 2026-09-26 21:22:54 on dist-test-slave-4d5t"
server_key: "8fbbd56ec2fb02b94ddcdb330083a698"
server_key_iv: "2fc4d1e2d700c5ed93791631cb9886f5"
server_key_version: "encryptionkey@0"
I20260926 21:22:54.165581 16240 heartbeater.cc:503] Master 127.14.142.62:38717 was elected leader, sending a full tablet report...
I20260926 21:22:54.167308 16383 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:22:54.171643 16398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:54.173409 16383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:22:54.173599 16383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "77ffdb46a34f4f7697acc91af7baef57"
format_stamp: "Formatted at 2026-09-26 21:22:54 on dist-test-slave-4d5t"
server_key: "8fbbd56ec2fb02b94ddcdb330083a698"
server_key_iv: "2fc4d1e2d700c5ed93791631cb9886f5"
server_key_version: "encryptionkey@0"
I20260926 21:22:54.173851 16383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:54.192739 16383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:54.197491 16383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:54.197784 16383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:54.200822 16383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:54.200948 16383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:54.201105 16383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:54.201186 16383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:22:54 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:54.229990 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.201763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48741 (local address 127.14.142.62:38717)
0926 21:22:54.202341 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.202348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.204075 (+ 1727us) server_negotiation.cc:407] Connection header received
0926 21:22:54.206487 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.206494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.206607 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.206733 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:54.209363 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.210922 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.213766 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.214506 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.214663 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.219986 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:54.219998 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:54.220008 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:54.220077 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:54.225274 (+ 5197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:54.226301 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:54.226309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:54.226313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:54.226497 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:54.227048 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:54.227054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:54.227057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:54.227455 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:54.227623 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.228843 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.229261 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":320,"threads_started":1}
I20260926 21:22:54.230273 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.202515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.14.142.3:48741)
0926 21:22:54.203784 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.203899 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.206095 (+ 2196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.207005 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.207014 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.207819 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:54.209039 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.209094 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.211195 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.211204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.213553 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.213577 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.214819 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.214830 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.215311 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.216274 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:54.216297 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:54.219613 (+ 3316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:54.225470 (+ 5857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:54.225481 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:54.225506 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:54.226150 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:54.226646 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:54.226652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:54.226660 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:54.226908 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:54.227595 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:54.227604 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:54.228188 (+ 584us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.228536 (+ 348us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.229082 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":613,"threads_started":1}
I20260926 21:22:54.253234 16383 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:34555
I20260926 21:22:54.253237 16518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:34555 every 8 connection(s)
I20260926 21:22:54.254608 16383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:22:54.256399 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.239229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.14.142.3:40095)
0926 21:22:54.241592 (+ 2363us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.241622 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.241813 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.242243 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.242249 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.242923 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:54.243260 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.243270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.244477 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.244483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.246272 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.246285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.247357 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.247364 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.247542 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.248217 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:54.248235 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:54.249245 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:54.253468 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:54.253476 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:54.253481 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:54.253895 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:54.254263 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:54.254268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:54.254272 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:54.254506 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:54.255107 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:54.255113 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:54.255325 (+ 212us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.255495 (+ 170us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.255918 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2118,"thread_start_us":1929,"threads_started":1}
I20260926 21:22:54.257040 16519 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38717
I20260926 21:22:54.257277 16519 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:54.257382 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.239164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:40095 (local address 127.14.142.62:38717)
0926 21:22:54.239405 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.239413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.241739 (+ 2326us) server_negotiation.cc:407] Connection header received
0926 21:22:54.241991 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.241998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.242093 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.242207 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:54.243435 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.244332 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.246421 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.247038 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.247315 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.249465 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:54.249473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:54.249478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:54.249525 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:54.253269 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:54.254028 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:54.254031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:54.254033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:54.254144 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:54.254661 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:54.254666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:54.254669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:54.254973 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:54.255146 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.256750 (+ 1604us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.257030 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:22:54.258031 16519 heartbeater.cc:511] Master 127.14.142.62:38717 requested a full tablet report, sending...
I20260926 21:22:54.259593 16032 ts_manager.cc:194] Registered new tserver with Master: 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555)
I20260926 21:22:54.260720 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16383
I20260926 21:22:54.260732 16032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:48741
I20260926 21:22:54.260922 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:22:54.261443 14904 external_mini_cluster.cc:1428] Setting key a591ff44e8d1289367f6f1192aa98cb2
I20260926 21:22:54.264801 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:22:54 dist-test-slave-4d5t krb5kdc[15987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 21:22:54 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:54.316001 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.300430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.14.142.62:38717)
0926 21:22:54.300827 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.300834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.300865 (+ 31us) server_negotiation.cc:407] Connection header received
0926 21:22:54.300956 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.300961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.301043 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.301167 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:54.302644 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.303401 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.305282 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.305893 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.306052 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.310694 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:54.310699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:54.310702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:54.310737 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:54.313369 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:54.314016 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:54.314021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:54.314023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:54.314157 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:54.314554 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:54.314558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:54.314560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:54.314871 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:54.315107 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.315445 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.315621 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 21:22:54.320140 16032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:22:54.327153 16032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:22:54.363309 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.341102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.14.142.2:38555)
0926 21:22:54.353969 (+ 12867us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.353980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.354017 (+ 37us) server_negotiation.cc:407] Connection header received
0926 21:22:54.354163 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.354174 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.354473 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.354705 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:54.355465 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.356498 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.360170 (+ 3672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.362000 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.362168 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.362393 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.362750 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.362979 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12656,"thread_start_us":13765,"threads_started":1}
I20260926 21:22:54.363417 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.340330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38555 (local address 127.0.0.1:45934)
0926 21:22:54.341987 (+ 1657us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.342039 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.342255 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.354706 (+ 12451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.354713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.354780 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:54.355291 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.355304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.356666 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.356673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.359874 (+ 3201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.359902 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.362241 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.362248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.362459 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.362520 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.362675 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.362888 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1322,"mutex_wait_us":128,"spinlock_wait_cycles":6912,"thread_start_us":12993,"threads_started":1}
I20260926 21:22:54.364015 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.342501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34555 (local address 127.0.0.1:56268)
0926 21:22:54.354538 (+ 12037us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.354560 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.354718 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.355881 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.355887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.355917 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:54.356327 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.356341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.358615 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.358620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.360952 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.360963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.362584 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.362590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.362764 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.363428 (+ 664us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.363522 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.363646 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11880,"mutex_wait_us":39,"thread_start_us":11722,"threads_started":1}
I20260926 21:22:54.364638 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.340335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46525 (local address 127.0.0.1:41380)
0926 21:22:54.353692 (+ 13357us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.353712 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.353901 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.354722 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.354727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.354769 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:54.355570 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.355590 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.358230 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.358242 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.360745 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.360762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.363176 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.363182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.363984 (+ 802us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.364026 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.364117 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.364246 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13186,"mutex_wait_us":1247,"spinlock_wait_cycles":1920,"thread_start_us":513,"threads_started":1}
I20260926 21:22:54.364806 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.342868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.14.142.3:34555)
0926 21:22:54.352587 (+ 9719us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.352599 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.354865 (+ 2266us) server_negotiation.cc:407] Connection header received
0926 21:22:54.354993 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.355003 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.355279 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.355505 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:54.356611 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.358417 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.361113 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.362056 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.362222 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.362460 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.364228 (+ 1768us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.364460 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9488,"thread_start_us":9241,"threads_started":1}
I20260926 21:22:54.366084 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.340794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.14.142.1:46525)
0926 21:22:54.351964 (+ 11170us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.351977 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.353841 (+ 1864us) server_negotiation.cc:407] Connection header received
0926 21:22:54.354133 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.354149 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.354477 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.355795 (+ 1318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:54.356070 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.358057 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.361050 (+ 2993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.362680 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.362841 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.363020 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.364898 (+ 1878us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.365789 (+ 891us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10938,"thread_start_us":12106,"threads_started":1}
I20260926 21:22:54.367839 16449 tablet_service.cc:1467] Processing CreateTablet for tablet efeeef2ce33d4c1ba13731f910a82570 (DEFAULT_TABLE table=test-table [id=2757f68188564150ae329241ab2b8926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:54.367856 16309 tablet_service.cc:1467] Processing CreateTablet for tablet efeeef2ce33d4c1ba13731f910a82570 (DEFAULT_TABLE table=test-table [id=2757f68188564150ae329241ab2b8926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:54.369294 16449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efeeef2ce33d4c1ba13731f910a82570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:54.369324 16309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efeeef2ce33d4c1ba13731f910a82570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:54.369469 16170 tablet_service.cc:1467] Processing CreateTablet for tablet efeeef2ce33d4c1ba13731f910a82570 (DEFAULT_TABLE table=test-table [id=2757f68188564150ae329241ab2b8926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:54.371078 16170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efeeef2ce33d4c1ba13731f910a82570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:54.381814 16538 tablet_bootstrap.cc:492] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Bootstrap starting.
I20260926 21:22:54.383847 16539 tablet_bootstrap.cc:492] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Bootstrap starting.
I20260926 21:22:54.385538 16540 tablet_bootstrap.cc:492] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Bootstrap starting.
I20260926 21:22:54.386044 16538 tablet_bootstrap.cc:654] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:54.387399 16539 tablet_bootstrap.cc:654] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:54.387439 16538 log.cc:824] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:54.388473 16540 tablet_bootstrap.cc:654] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:54.388607 16539 log.cc:824] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:54.389719 16540 log.cc:824] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:54.391572 16538 tablet_bootstrap.cc:492] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: No bootstrap required, opened a new log
I20260926 21:22:54.391569 16539 tablet_bootstrap.cc:492] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: No bootstrap required, opened a new log
I20260926 21:22:54.391937 16538 ts_tablet_manager.cc:1397] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260926 21:22:54.391937 16539 ts_tablet_manager.cc:1397] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260926 21:22:54.392360 16540 tablet_bootstrap.cc:492] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: No bootstrap required, opened a new log
I20260926 21:22:54.392673 16540 ts_tablet_manager.cc:1397] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260926 21:22:54.400008 16540 raft_consensus.cc:348] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.400427 16540 raft_consensus.cc:374] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:54.400521 16540 raft_consensus.cc:729] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a5c24d80234837ac3ae14ddbcd1ee8, State: Initialized, Role: FOLLOWER
I20260926 21:22:54.401221 16540 consensus_queue.cc:260] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.403159 16540 ts_tablet_manager.cc:1428] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260926 21:22:54.403170 16539 raft_consensus.cc:348] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.403609 16539 raft_consensus.cc:374] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:54.403717 16539 raft_consensus.cc:729] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6125a01dbbc42649ea2ae5c05a8673e, State: Initialized, Role: FOLLOWER
I20260926 21:22:54.403185 16538 raft_consensus.cc:348] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.404109 16538 raft_consensus.cc:374] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:54.404352 16538 raft_consensus.cc:729] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ffdb46a34f4f7697acc91af7baef57, State: Initialized, Role: FOLLOWER
I20260926 21:22:54.405015 16539 consensus_queue.cc:260] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
W20260926 21:22:54.405570 16241 tablet.cc:2369] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:22:54.405400 16538 consensus_queue.cc:260] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.409377 16519 heartbeater.cc:503] Master 127.14.142.62:38717 was elected leader, sending a full tablet report...
I20260926 21:22:54.410001 16379 heartbeater.cc:503] Master 127.14.142.62:38717 was elected leader, sending a full tablet report...
I20260926 21:22:54.410276 16538 ts_tablet_manager.cc:1428] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Time spent starting tablet: real 0.018s user 0.014s sys 0.003s
I20260926 21:22:54.410388 16539 ts_tablet_manager.cc:1428] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
W20260926 21:22:54.452044 16380 tablet.cc:2369] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:22:54.506500 16520 tablet.cc:2369] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:22:54.535547 16544 raft_consensus.cc:482] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:22:54.535943 16544 raft_consensus.cc:504] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.538055 16544 leader_election.cc:288] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555), e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555)
I20260926 21:22:54.545486 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.538451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:55295 (local address 127.14.142.3:34555)
0926 21:22:54.538666 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.538672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.539558 (+ 886us) server_negotiation.cc:407] Connection header received
0926 21:22:54.539753 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.539758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.539821 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.539936 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:54.540626 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.541517 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.543549 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.544406 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.544654 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.544755 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.545047 (+ 292us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.545188 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:22:54.545780 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.538551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:55107 (local address 127.14.142.2:38555)
0926 21:22:54.538809 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.538819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.539459 (+ 640us) server_negotiation.cc:407] Connection header received
0926 21:22:54.539699 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.539704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.539764 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.539868 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:54.540455 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.541247 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.543577 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.544504 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.544986 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.545068 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.545362 (+ 294us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.545529 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:22:54.546090 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.538622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34555 (local address 127.14.142.1:55295)
0926 21:22:54.539452 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.539467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.539586 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.539953 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.539957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.539989 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:54.540436 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.540445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.541701 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.541709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.543357 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.543385 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.544665 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.544671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.544859 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.544886 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.545021 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.545172 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":6,"thread_start_us":509,"threads_started":1}
I20260926 21:22:54.546123 16547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.538413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38555 (local address 127.14.142.1:55107)
0926 21:22:54.539318 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.539343 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.539556 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.539890 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.539894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.539919 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:54.540304 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.540316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.541380 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.541387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.543231 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:54.543264 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.545002 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.545009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:54.545184 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.545208 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 21:22:54.545328 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 21:22:54.545456 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":37,"spinlock_wait_cycles":896,"thread_start_us":483,"threads_started":1}
I20260926 21:22:54.547175 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efeeef2ce33d4c1ba13731f910a82570" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" is_pre_election: true
I20260926 21:22:54.547199 16469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efeeef2ce33d4c1ba13731f910a82570" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ffdb46a34f4f7697acc91af7baef57" is_pre_election: true
I20260926 21:22:54.547684 16334 raft_consensus.cc:2454] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 0.
I20260926 21:22:54.547688 16469 raft_consensus.cc:2454] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 0.
I20260926 21:22:54.548379 16120 leader_election.cc:302] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a5c24d80234837ac3ae14ddbcd1ee8, e6125a01dbbc42649ea2ae5c05a8673e; no voters:
I20260926 21:22:54.548806 16544 raft_consensus.cc:2781] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:22:54.548976 16544 raft_consensus.cc:482] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:22:54.549089 16544 raft_consensus.cc:3037] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:54.551352 16544 raft_consensus.cc:504] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.552059 16544 leader_election.cc:288] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 election: Requested vote from peers 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555), e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555)
I20260926 21:22:54.552654 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efeeef2ce33d4c1ba13731f910a82570" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6125a01dbbc42649ea2ae5c05a8673e"
I20260926 21:22:54.552733 16469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efeeef2ce33d4c1ba13731f910a82570" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ffdb46a34f4f7697acc91af7baef57"
I20260926 21:22:54.552925 16334 raft_consensus.cc:3037] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:54.553004 16469 raft_consensus.cc:3037] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:54.555173 16334 raft_consensus.cc:2454] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 1.
I20260926 21:22:54.555630 16120 leader_election.cc:302] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a5c24d80234837ac3ae14ddbcd1ee8, e6125a01dbbc42649ea2ae5c05a8673e; no voters:
I20260926 21:22:54.555688 16469 raft_consensus.cc:2454] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 1.
I20260926 21:22:54.555931 16544 raft_consensus.cc:2781] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:54.556492 16544 raft_consensus.cc:686] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 LEADER]: Becoming Leader. State: Replica: 38a5c24d80234837ac3ae14ddbcd1ee8, State: Running, Role: LEADER
I20260926 21:22:54.557322 16544 consensus_queue.cc:237] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } }
I20260926 21:22:54.564670 16032 catalog_manager.cc:5495] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1). New cstate: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } health_report { overall_health: UNKNOWN } } }
I20260926 21:22:54.600737 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.593796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.14.142.1:46525)
0926 21:22:54.594012 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.594017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.594354 (+ 337us) server_negotiation.cc:407] Connection header received
0926 21:22:54.594659 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.594663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.594760 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.594930 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:22:54.595540 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.596312 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.598086 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:54.598571 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:54.598688 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:54.599414 (+ 726us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:22:54.599986 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:22:54.600103 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:54.600321 (+ 218us) server_negotiation.cc:299] Negotiation successful
0926 21:22:54.600396 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:22:54.609922 16469 raft_consensus.cc:1261] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Refusing update from remote peer 38a5c24d80234837ac3ae14ddbcd1ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:22:54.609920 16334 raft_consensus.cc:1261] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Refusing update from remote peer 38a5c24d80234837ac3ae14ddbcd1ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:22:54.610913 16544 consensus_queue.cc:1035] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:22:54.611227 16549 consensus_queue.cc:1035] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:22:54.625227 16552 mvcc.cc:205] Tried to move back new op lower bound from 7333715044792029184 to 7333715044592775168. Current Snapshot: MvccSnapshot[applied={T|T < 7333715044792029184}]
I20260926 21:22:54.649765 16032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:22:54.650058 16032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:22:54.653805 16032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: Sending DeleteTablet for 3 replicas of tablet efeeef2ce33d4c1ba13731f910a82570
I20260926 21:22:54.655009 16170 tablet_service.cc:1514] Processing DeleteTablet for tablet efeeef2ce33d4c1ba13731f910a82570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:22:54 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41380
I20260926 21:22:54.655226 16449 tablet_service.cc:1514] Processing DeleteTablet for tablet efeeef2ce33d4c1ba13731f910a82570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:22:54 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:56268
I20260926 21:22:54.655406 16309 tablet_service.cc:1514] Processing DeleteTablet for tablet efeeef2ce33d4c1ba13731f910a82570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:22:54 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:45934
I20260926 21:22:54.657959 16032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:22:54.659272 16032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:22:54.666476 16564 tablet_replica.cc:330] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: stopping tablet replica
I20260926 21:22:54.667779 16564 raft_consensus.cc:2229] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:22:54.668706 16564 raft_consensus.cc:2258] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:22:54.668865 16565 tablet_replica.cc:330] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: stopping tablet replica
I20260926 21:22:54.669289 16449 tablet_service.cc:1467] Processing CreateTablet for tablet 74ae5273a24340d3818624e56f6a0f9b (DEFAULT_TABLE table=test-table [id=bdc5fc0954144791b199083fc72867e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:54.669355 16565 raft_consensus.cc:2229] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:22:54.669688 16308 tablet_service.cc:1467] Processing CreateTablet for tablet 74ae5273a24340d3818624e56f6a0f9b (DEFAULT_TABLE table=test-table [id=bdc5fc0954144791b199083fc72867e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:54.669818 16449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74ae5273a24340d3818624e56f6a0f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:54.669945 16565 raft_consensus.cc:2258] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:22:54.670357 16308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74ae5273a24340d3818624e56f6a0f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:54.672451 16565 ts_tablet_manager.cc:1905] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:22:54.673532 16169 tablet_service.cc:1467] Processing CreateTablet for tablet 74ae5273a24340d3818624e56f6a0f9b (DEFAULT_TABLE table=test-table [id=bdc5fc0954144791b199083fc72867e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:54.674129 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74ae5273a24340d3818624e56f6a0f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:54.675072 16564 ts_tablet_manager.cc:1905] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:22:54.676800 16538 tablet_bootstrap.cc:492] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Bootstrap starting.
I20260926 21:22:54.678511 16538 tablet_bootstrap.cc:654] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:54.681676 16563 tablet_replica.cc:330] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: stopping tablet replica
I20260926 21:22:54.682163 16540 tablet_bootstrap.cc:492] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Bootstrap starting.
I20260926 21:22:54.682272 16563 raft_consensus.cc:2229] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:22:54.683097 16563 raft_consensus.cc:2258] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:22:54.684238 16540 tablet_bootstrap.cc:654] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:54.686100 16564 ts_tablet_manager.cc:1918] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:22:54.686292 16564 log.cc:1196] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/efeeef2ce33d4c1ba13731f910a82570
I20260926 21:22:54.686872 16564 ts_tablet_manager.cc:1939] T efeeef2ce33d4c1ba13731f910a82570 P 77ffdb46a34f4f7697acc91af7baef57: Deleting consensus metadata
I20260926 21:22:54.691218 16539 tablet_bootstrap.cc:492] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Bootstrap starting.
I20260926 21:22:54.693118 16539 tablet_bootstrap.cc:654] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:54.696748 16563 ts_tablet_manager.cc:1905] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:22:54.697501 16565 ts_tablet_manager.cc:1918] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:22:54.697713 16565 log.cc:1196] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/efeeef2ce33d4c1ba13731f910a82570
I20260926 21:22:54.698333 16565 ts_tablet_manager.cc:1939] T efeeef2ce33d4c1ba13731f910a82570 P e6125a01dbbc42649ea2ae5c05a8673e: Deleting consensus metadata
I20260926 21:22:54.700487 16016 catalog_manager.cc:4841] TS e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555): tablet efeeef2ce33d4c1ba13731f910a82570 (table test-table [id=2757f68188564150ae329241ab2b8926]) successfully deleted
I20260926 21:22:54.701521 16018 catalog_manager.cc:4841] TS 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555): tablet efeeef2ce33d4c1ba13731f910a82570 (table test-table [id=2757f68188564150ae329241ab2b8926]) successfully deleted
I20260926 21:22:54.702384 16539 tablet_bootstrap.cc:492] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: No bootstrap required, opened a new log
I20260926 21:22:54.702545 16539 ts_tablet_manager.cc:1397] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260926 21:22:54.703615 16539 raft_consensus.cc:348] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.703860 16539 raft_consensus.cc:374] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:54.703963 16539 raft_consensus.cc:729] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6125a01dbbc42649ea2ae5c05a8673e, State: Initialized, Role: FOLLOWER
I20260926 21:22:54.704355 16539 consensus_queue.cc:260] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.705210 16563 ts_tablet_manager.cc:1918] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:22:54.705415 16563 log.cc:1196] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/efeeef2ce33d4c1ba13731f910a82570
I20260926 21:22:54.705880 16539 ts_tablet_manager.cc:1428] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 21:22:54.706082 16563 ts_tablet_manager.cc:1939] T efeeef2ce33d4c1ba13731f910a82570 P 38a5c24d80234837ac3ae14ddbcd1ee8: Deleting consensus metadata
I20260926 21:22:54.708030 16018 catalog_manager.cc:4841] TS 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1:46525): tablet efeeef2ce33d4c1ba13731f910a82570 (table test-table [id=2757f68188564150ae329241ab2b8926]) successfully deleted
I20260926 21:22:54.714954 16538 tablet_bootstrap.cc:492] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: No bootstrap required, opened a new log
I20260926 21:22:54.715142 16538 ts_tablet_manager.cc:1397] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.010s
I20260926 21:22:54.715158 16540 tablet_bootstrap.cc:492] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: No bootstrap required, opened a new log
I20260926 21:22:54.715404 16540 ts_tablet_manager.cc:1397] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.011s
I20260926 21:22:54.715989 16538 raft_consensus.cc:348] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.716221 16538 raft_consensus.cc:374] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:54.716303 16538 raft_consensus.cc:729] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ffdb46a34f4f7697acc91af7baef57, State: Initialized, Role: FOLLOWER
I20260926 21:22:54.716441 16540 raft_consensus.cc:348] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.716729 16540 raft_consensus.cc:374] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:54.716607 16538 consensus_queue.cc:260] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.716842 16540 raft_consensus.cc:729] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a5c24d80234837ac3ae14ddbcd1ee8, State: Initialized, Role: FOLLOWER
I20260926 21:22:54.717154 16540 consensus_queue.cc:260] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.718106 16538 ts_tablet_manager.cc:1428] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 21:22:54.720244 16540 ts_tablet_manager.cc:1428] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260926 21:22:54.994799 16546 raft_consensus.cc:482] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:22:54.995064 16546 raft_consensus.cc:504] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:54.997207 16546 leader_election.cc:288] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1:46525), e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555)
I20260926 21:22:55.007385 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.998195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:35885 (local address 127.14.142.2:38555)
0926 21:22:54.998400 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.998409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.999978 (+ 1569us) server_negotiation.cc:407] Connection header received
0926 21:22:55.000214 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.000218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.000286 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.000391 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:55.001020 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.001831 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.004983 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.006362 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.006565 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.006650 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.006911 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.007048 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 21:22:55.007438 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.997662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38555 (local address 127.14.142.3:35885)
0926 21:22:54.999845 (+ 2183us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.999872 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:55.000053 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:55.000405 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:55.000410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:55.000450 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:55.000847 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.000857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.002103 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.002110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.004737 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.004757 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.006562 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.006567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.006700 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.006727 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 21:22:55.006881 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 21:22:55.007042 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2059,"mutex_wait_us":747,"thread_start_us":1226,"threads_started":1}
I20260926 21:22:55.008750 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ae5273a24340d3818624e56f6a0f9b" candidate_uuid: "77ffdb46a34f4f7697acc91af7baef57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" is_pre_election: true
I20260926 21:22:55.009047 16334 raft_consensus.cc:2454] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77ffdb46a34f4f7697acc91af7baef57 in term 0.
I20260926 21:22:55.009464 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.997916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:58343 (local address 127.14.142.1:46525)
0926 21:22:54.998168 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:54.998176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:54.999098 (+ 922us) server_negotiation.cc:407] Connection header received
0926 21:22:54.999402 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:54.999409 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:54.999567 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:54.999672 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:55.000583 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.002055 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.006780 (+ 4725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.008210 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.008442 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.008530 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.008924 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.009074 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 21:22:55.009610 16399 leader_election.cc:302] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77ffdb46a34f4f7697acc91af7baef57, e6125a01dbbc42649ea2ae5c05a8673e; no voters:
I20260926 21:22:55.009850 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:54.997662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46525 (local address 127.14.142.3:58343)
0926 21:22:54.998961 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:54.998980 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:54.999225 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:54.999895 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:54.999902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:54.999956 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:55.000274 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.000289 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.002268 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.002274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.004747 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.004757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.008467 (+ 3710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.008473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.008718 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.008759 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 21:22:55.008863 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 21:22:55.009248 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"mutex_wait_us":77,"thread_start_us":941,"threads_started":1}
I20260926 21:22:55.010138 16546 raft_consensus.cc:2781] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:22:55.010295 16546 raft_consensus.cc:482] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:22:55.010864 16195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ae5273a24340d3818624e56f6a0f9b" candidate_uuid: "77ffdb46a34f4f7697acc91af7baef57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" is_pre_election: true
I20260926 21:22:55.011438 16195 raft_consensus.cc:2454] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77ffdb46a34f4f7697acc91af7baef57 in term 0.
I20260926 21:22:55.010417 16546 raft_consensus.cc:3037] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:55.016350 16546 raft_consensus.cc:504] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.017414 16546 leader_election.cc:288] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [CANDIDATE]: Term 1 election: Requested vote from peers 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1:46525), e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555)
I20260926 21:22:55.017973 16195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ae5273a24340d3818624e56f6a0f9b" candidate_uuid: "77ffdb46a34f4f7697acc91af7baef57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8"
I20260926 21:22:55.018060 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74ae5273a24340d3818624e56f6a0f9b" candidate_uuid: "77ffdb46a34f4f7697acc91af7baef57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6125a01dbbc42649ea2ae5c05a8673e"
I20260926 21:22:55.018215 16195 raft_consensus.cc:3037] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:55.018356 16334 raft_consensus.cc:3037] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:55.021330 16195 raft_consensus.cc:2454] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77ffdb46a34f4f7697acc91af7baef57 in term 1.
I20260926 21:22:55.021957 16401 leader_election.cc:302] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a5c24d80234837ac3ae14ddbcd1ee8, 77ffdb46a34f4f7697acc91af7baef57; no voters:
I20260926 21:22:55.022477 16334 raft_consensus.cc:2454] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77ffdb46a34f4f7697acc91af7baef57 in term 1.
I20260926 21:22:55.023085 16546 raft_consensus.cc:2781] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:55.023693 16546 raft_consensus.cc:686] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 1 LEADER]: Becoming Leader. State: Replica: 77ffdb46a34f4f7697acc91af7baef57, State: Running, Role: LEADER
I20260926 21:22:55.024233 16546 consensus_queue.cc:237] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.028851 16030 catalog_manager.cc:5495] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "77ffdb46a34f4f7697acc91af7baef57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } health_report { overall_health: HEALTHY } } }
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457772, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:55.110991 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.091244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38717 (local address 127.0.0.1:48336)
0926 21:22:55.092204 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:55.092221 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:55.092358 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:55.092771 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:55.092774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:55.093487 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:55.093697 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.093706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.095341 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.095345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.096681 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.096696 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.098094 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.098102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.098282 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.099224 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:55.099246 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:55.102555 (+ 3309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:55.107773 (+ 5218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:55.107788 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:55.107796 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:55.108443 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:55.108883 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:55.108890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:55.108894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:55.109059 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:55.109728 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:55.109738 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:55.109914 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 21:22:55.110098 (+ 184us) client_negotiation.cc:240] Negotiation successful
0926 21:22:55.110429 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":661,"threads_started":1}
I20260926 21:22:55.111039 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.091244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.14.142.62:38717)
0926 21:22:55.092009 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.092019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.092320 (+ 301us) server_negotiation.cc:407] Connection header received
0926 21:22:55.092479 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.092486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.092603 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.092733 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:55.093878 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.095166 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.096843 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.097721 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.097865 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.102819 (+ 4954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:55.102826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:55.102831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:55.103006 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:55.107549 (+ 4543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.108600 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.108607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.108614 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.108760 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.109172 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.109180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.109183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.109573 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:55.109729 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.110189 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.110566 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":561,"thread_start_us":377,"threads_started":1}
I20260926 21:22:55.117079 16030 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:48336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 21:22:55.118767 16030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:22:55.128976 16169 tablet_service.cc:1467] Processing CreateTablet for tablet e5b3f5ce050b43779cdfef71c34f02f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6389fa223d3c422da4c5339019fa3a04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:22:55.129004 16308 tablet_service.cc:1467] Processing CreateTablet for tablet e5b3f5ce050b43779cdfef71c34f02f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6389fa223d3c422da4c5339019fa3a04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:22:55.129565 16169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b3f5ce050b43779cdfef71c34f02f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:55.129573 16308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b3f5ce050b43779cdfef71c34f02f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:55.132474 16449 tablet_service.cc:1467] Processing CreateTablet for tablet e5b3f5ce050b43779cdfef71c34f02f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6389fa223d3c422da4c5339019fa3a04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:22:55.133153 16449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b3f5ce050b43779cdfef71c34f02f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:55.138084 16540 tablet_bootstrap.cc:492] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: Bootstrap starting.
I20260926 21:22:55.139981 16540 tablet_bootstrap.cc:654] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:55.140380 16539 tablet_bootstrap.cc:492] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: Bootstrap starting.
I20260926 21:22:55.141988 16538 tablet_bootstrap.cc:492] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: Bootstrap starting.
I20260926 21:22:55.142601 16539 tablet_bootstrap.cc:654] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:55.144223 16538 tablet_bootstrap.cc:654] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:55.154577 16539 tablet_bootstrap.cc:492] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: No bootstrap required, opened a new log
I20260926 21:22:55.154954 16539 ts_tablet_manager.cc:1397] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260926 21:22:55.156109 16539 raft_consensus.cc:348] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.156597 16539 raft_consensus.cc:374] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:55.156808 16539 raft_consensus.cc:729] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6125a01dbbc42649ea2ae5c05a8673e, State: Initialized, Role: FOLLOWER
I20260926 21:22:55.157228 16539 consensus_queue.cc:260] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.157817 16538 tablet_bootstrap.cc:492] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: No bootstrap required, opened a new log
I20260926 21:22:55.158033 16538 ts_tablet_manager.cc:1397] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.009s
I20260926 21:22:55.158157 16539 ts_tablet_manager.cc:1428] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260926 21:22:55.157963 16545 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.158417 16545 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.160284 16538 raft_consensus.cc:348] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.161079 16538 raft_consensus.cc:374] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:55.161197 16538 raft_consensus.cc:729] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ffdb46a34f4f7697acc91af7baef57, State: Initialized, Role: FOLLOWER
I20260926 21:22:55.161489 16538 consensus_queue.cc:260] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.162403 16538 ts_tablet_manager.cc:1428] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260926 21:22:55.162333 16546 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.162726 16546 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.164206 16540 tablet_bootstrap.cc:492] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: No bootstrap required, opened a new log
I20260926 21:22:55.164426 16540 ts_tablet_manager.cc:1397] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.016s
I20260926 21:22:55.165257 16540 raft_consensus.cc:348] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.165483 16540 raft_consensus.cc:374] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:55.165570 16540 raft_consensus.cc:729] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a5c24d80234837ac3ae14ddbcd1ee8, State: Initialized, Role: FOLLOWER
I20260926 21:22:55.165832 16540 consensus_queue.cc:260] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.166457 16540 ts_tablet_manager.cc:1428] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 21:22:55.166290 16559 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.166613 16559 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.225260 16559 raft_consensus.cc:482] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:22:55.225462 16559 raft_consensus.cc:504] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.226377 16559 leader_election.cc:288] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555), 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555)
I20260926 21:22:55.227033 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b3f5ce050b43779cdfef71c34f02f3" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" is_pre_election: true
I20260926 21:22:55.227067 16469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b3f5ce050b43779cdfef71c34f02f3" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ffdb46a34f4f7697acc91af7baef57" is_pre_election: true
I20260926 21:22:55.227351 16334 raft_consensus.cc:2454] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 0.
I20260926 21:22:55.227326 16469 raft_consensus.cc:2454] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 0.
I20260926 21:22:55.227808 16122 leader_election.cc:302] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a5c24d80234837ac3ae14ddbcd1ee8, 77ffdb46a34f4f7697acc91af7baef57; no voters:
I20260926 21:22:55.228108 16559 raft_consensus.cc:2781] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:22:55.228250 16559 raft_consensus.cc:482] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:22:55.228375 16559 raft_consensus.cc:3037] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:55.230736 16559 raft_consensus.cc:504] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.231390 16559 leader_election.cc:288] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 election: Requested vote from peers e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555), 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555)
I20260926 21:22:55.231997 16469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b3f5ce050b43779cdfef71c34f02f3" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ffdb46a34f4f7697acc91af7baef57"
I20260926 21:22:55.231979 16334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b3f5ce050b43779cdfef71c34f02f3" candidate_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6125a01dbbc42649ea2ae5c05a8673e"
I20260926 21:22:55.232237 16469 raft_consensus.cc:3037] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:55.232275 16334 raft_consensus.cc:3037] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:55.234480 16334 raft_consensus.cc:2454] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 1.
I20260926 21:22:55.234812 16469 raft_consensus.cc:2454] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38a5c24d80234837ac3ae14ddbcd1ee8 in term 1.
I20260926 21:22:55.234970 16120 leader_election.cc:302] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a5c24d80234837ac3ae14ddbcd1ee8, e6125a01dbbc42649ea2ae5c05a8673e; no voters:
I20260926 21:22:55.235298 16559 raft_consensus.cc:2781] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:55.235765 16559 raft_consensus.cc:686] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 LEADER]: Becoming Leader. State: Replica: 38a5c24d80234837ac3ae14ddbcd1ee8, State: Running, Role: LEADER
I20260926 21:22:55.236090 16559 consensus_queue.cc:237] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } }
I20260926 21:22:55.238140 16583 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: TxnStatusTablet state changed. Reason: New leader 38a5c24d80234837ac3ae14ddbcd1ee8. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.238480 16583 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:22:55.239243 16585 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 21:22:55.239503 16585 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 21:22:55.239631 16032 catalog_manager.cc:5495] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1). New cstate: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } health_report { overall_health: UNKNOWN } } }
I20260926 21:22:55.281765 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.275507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46525 (local address 127.0.0.1:41394)
0926 21:22:55.275773 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:55.275795 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:55.275940 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:55.276481 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:55.276486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:55.276499 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:22:55.276737 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.276752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.278231 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.278236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.279456 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.279468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.280457 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.280462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.280682 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.280747 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.281227 (+ 480us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.281248 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 21:22:55.281346 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 21:22:55.281481 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 21:22:55.281796 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.275933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.14.142.1:46525)
0926 21:22:55.276138 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.276144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.276163 (+ 19us) server_negotiation.cc:407] Connection header received
0926 21:22:55.276241 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.276246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.276344 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.276446 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:22:55.276896 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.278098 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.279594 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.280297 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.280429 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.280942 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.281085 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.281189 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.281386 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.281479 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 21:22:55.286303 16334 raft_consensus.cc:1261] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Refusing update from remote peer 38a5c24d80234837ac3ae14ddbcd1ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:22:55.286311 16469 raft_consensus.cc:1261] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Refusing update from remote peer 38a5c24d80234837ac3ae14ddbcd1ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:22:55.287003 16583 consensus_queue.cc:1035] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:22:55.287355 16559 consensus_queue.cc:1035] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:22:55.296048 16546 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: TxnStatusTablet state changed. Reason: New leader 38a5c24d80234837ac3ae14ddbcd1ee8. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.296468 16546 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.299611 16589 mvcc.cc:205] Tried to move back new op lower bound from 7333715047566221312 to 7333715047370055680. Current Snapshot: MvccSnapshot[applied={T|T < 7333715047566221312}]
I20260926 21:22:55.301064 16545 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: TxnStatusTablet state changed. Reason: New leader 38a5c24d80234837ac3ae14ddbcd1ee8. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.301469 16545 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.302882 16559 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.304952 16559 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:22:55.305712 16545 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.305958 16545 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P e6125a01dbbc42649ea2ae5c05a8673e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.304036 16546 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.307117 16546 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 77ffdb46a34f4f7697acc91af7baef57: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:55.307376 16559 tablet_replica.cc:439] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 } } peers { permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 } } peers { permanent_uuid: "77ffdb46a34f4f7697acc91af7baef57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34555 } } }
I20260926 21:22:55.307658 16559 tablet_replica.cc:442] T e5b3f5ce050b43779cdfef71c34f02f3 P 38a5c24d80234837ac3ae14ddbcd1ee8: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:22:55.327795 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.318487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.14.142.3:34555)
0926 21:22:55.319937 (+ 1450us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.319942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.319959 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:22:55.320052 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.320056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.320131 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.320285 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:22:55.321195 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.322508 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.324931 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.325551 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.325684 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.326369 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.327066 (+ 697us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.327181 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.327336 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.327414 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":1164,"threads_started":1}
I20260926 21:22:55.346835 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.340050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46525 (local address 127.14.142.3:51957)
0926 21:22:55.340862 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:55.340879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:55.341011 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:55.341380 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:55.341383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:55.341394 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:22:55.341688 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.341695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.343195 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.343199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.344435 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.344447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.345616 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.345621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.345744 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.345805 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.346312 (+ 507us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.346321 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 21:22:55.346395 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 21:22:55.346509 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":440,"threads_started":1}
I20260926 21:22:55.346876 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.340123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:51957 (local address 127.14.142.1:46525)
0926 21:22:55.340339 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.340345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.340963 (+ 618us) server_negotiation.cc:407] Connection header received
0926 21:22:55.341138 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.341145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.341234 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.341350 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:22:55.341828 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.343034 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.344643 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.345413 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.345566 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.345999 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.346196 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.346319 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.346445 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.346534 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 21:22:55.357760 16334 raft_consensus.cc:1261] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Refusing update from remote peer 77ffdb46a34f4f7697acc91af7baef57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:22:55.358058 16195 raft_consensus.cc:1261] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Refusing update from remote peer 77ffdb46a34f4f7697acc91af7baef57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:22:55.358491 16571 consensus_queue.cc:1035] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6125a01dbbc42649ea2ae5c05a8673e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:22:55.359014 16546 consensus_queue.cc:1035] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a5c24d80234837ac3ae14ddbcd1ee8" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:22:55.404137 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.398518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:48055 (local address 127.14.142.3:34555)
0926 21:22:55.398707 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.398713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.398894 (+ 181us) server_negotiation.cc:407] Connection header received
0926 21:22:55.399157 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.399163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.399243 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.399338 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:22:55.399972 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.400872 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.402411 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.402819 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.402945 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.403399 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.403498 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.403596 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.403736 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.403811 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:22:55.404640 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.398018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34555 (local address 127.14.142.1:48055)
0926 21:22:55.398793 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:55.398813 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:55.399013 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:55.399353 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:55.399362 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:55.399373 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:22:55.399784 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.399794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.401003 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.401007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.402273 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:55.402284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.403011 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.403017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:55.403154 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.403210 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:22:55.403597 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:22:55.403611 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 21:22:55.403713 (+ 102us) client_negotiation.cc:240] Negotiation successful
0926 21:22:55.403864 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":708,"threads_started":1}
I20260926 21:22:55.456178 16030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:22:55.456501 16030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:22:55.460884 16030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70a25af7a6d64ffe9c76d853767fae5c: Sending DeleteTablet for 3 replicas of tablet 74ae5273a24340d3818624e56f6a0f9b
I20260926 21:22:55.461794 16169 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ae5273a24340d3818624e56f6a0f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:22:55 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41380
I20260926 21:22:55.462836 16308 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ae5273a24340d3818624e56f6a0f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:22:55 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:45934
I20260926 21:22:55.466593 16449 tablet_service.cc:1514] Processing DeleteTablet for tablet 74ae5273a24340d3818624e56f6a0f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:22:55 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:56268
I20260926 21:22:55.473187 16613 tablet_replica.cc:330] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: stopping tablet replica
I20260926 21:22:55.473817 16613 raft_consensus.cc:2229] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:22:55.474880 16613 raft_consensus.cc:2258] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:22:55.476259 16613 ts_tablet_manager.cc:1905] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:22:55.498041 16612 tablet_replica.cc:330] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: stopping tablet replica
I20260926 21:22:55.500844 16612 raft_consensus.cc:2229] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:22:55.501303 16612 raft_consensus.cc:2258] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:22:55.502049 16611 tablet_replica.cc:330] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: stopping tablet replica
I20260926 21:22:55.503340 16612 ts_tablet_manager.cc:1905] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:22:55.503644 16611 raft_consensus.cc:2229] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:22:55.503981 16611 raft_consensus.cc:2258] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:22:55.505267 16613 ts_tablet_manager.cc:1918] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:22:55.505436 16613 log.cc:1196] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/74ae5273a24340d3818624e56f6a0f9b
I20260926 21:22:55.505949 16613 ts_tablet_manager.cc:1939] T 74ae5273a24340d3818624e56f6a0f9b P 77ffdb46a34f4f7697acc91af7baef57: Deleting consensus metadata
I20260926 21:22:55.506219 16611 ts_tablet_manager.cc:1905] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:22:55.507259 16018 catalog_manager.cc:4841] TS 77ffdb46a34f4f7697acc91af7baef57 (127.14.142.3:34555): tablet 74ae5273a24340d3818624e56f6a0f9b (table test-table [id=bdc5fc0954144791b199083fc72867e3]) successfully deleted
I20260926 21:22:55.526110 16612 ts_tablet_manager.cc:1918] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:22:55.536294 16612 log.cc:1196] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/74ae5273a24340d3818624e56f6a0f9b
I20260926 21:22:55.536969 16612 ts_tablet_manager.cc:1939] T 74ae5273a24340d3818624e56f6a0f9b P e6125a01dbbc42649ea2ae5c05a8673e: Deleting consensus metadata
I20260926 21:22:55.537500 16611 ts_tablet_manager.cc:1918] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:22:55.538972 16016 catalog_manager.cc:4841] TS e6125a01dbbc42649ea2ae5c05a8673e (127.14.142.2:38555): tablet 74ae5273a24340d3818624e56f6a0f9b (table test-table [id=bdc5fc0954144791b199083fc72867e3]) successfully deleted
I20260926 21:22:55.539359 16611 log.cc:1196] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/74ae5273a24340d3818624e56f6a0f9b
I20260926 21:22:55.539927 16611 ts_tablet_manager.cc:1939] T 74ae5273a24340d3818624e56f6a0f9b P 38a5c24d80234837ac3ae14ddbcd1ee8: Deleting consensus metadata
I20260926 21:22:55.541227 16018 catalog_manager.cc:4841] TS 38a5c24d80234837ac3ae14ddbcd1ee8 (127.14.142.1:46525): tablet 74ae5273a24340d3818624e56f6a0f9b (table test-table [id=bdc5fc0954144791b199083fc72867e3]) successfully deleted
I20260926 21:22:55.551707 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.499590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.14.142.1:46525)
0926 21:22:55.499895 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.499905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.509504 (+ 9599us) server_negotiation.cc:407] Connection header received
0926 21:22:55.514430 (+ 4926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.514437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.514543 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.514615 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:55.520633 (+ 6018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.521986 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.528266 (+ 6280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.529111 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.529228 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.543021 (+ 13793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:55.543042 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:55.543066 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:55.543123 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:55.547634 (+ 4511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.548760 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.548772 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.548777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.548969 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.549440 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.549446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.549451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.549981 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:55.550311 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.550973 (+ 662us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.551279 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260926 21:22:55.553694 16235 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35990
I20260926 21:22:55.574739 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.557804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.14.142.62:38717)
0926 21:22:55.558105 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.558113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.558478 (+ 365us) server_negotiation.cc:407] Connection header received
0926 21:22:55.558794 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.558802 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.558906 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.558985 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:55.561102 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.562348 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.564464 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.565246 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.565363 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.569047 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:55.569055 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:55.569060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:55.569109 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:55.572086 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.572819 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.572826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.572829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.572965 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.573331 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.573335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.573339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.573626 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:55.573888 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.574186 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.574403 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
W20260926 21:22:55.575344 16030 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53582
I20260926 21:22:55.577731 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16104
I20260926 21:22:55.598969 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.579947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.14.142.1:46525)
0926 21:22:55.580698 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.580707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.585146 (+ 4439us) server_negotiation.cc:407] Connection header received
0926 21:22:55.585357 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.585363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.585443 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.585563 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:55.586706 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.587503 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.588685 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.589204 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.589358 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.591622 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:55.591628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:55.591633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:55.591677 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:55.595764 (+ 4087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.596588 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.596597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.596604 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.596790 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.597225 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.597231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.597234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.597542 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:55.597868 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.598186 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.598482 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538}
W20260926 21:22:55.599614 16235 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36004
W20260926 21:22:55.600317 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16104: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:22:55.600543 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16104
I20260926 21:22:55.635356 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16244
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:22:55.653743 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.636573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.14.142.2:38555)
0926 21:22:55.637511 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.637516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.637537 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:22:55.637603 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.637607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.637690 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.637848 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:55.639261 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.640127 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.641900 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.642417 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.642581 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.647477 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:55.647493 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:55.647512 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:55.647556 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:55.650778 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.651624 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.651631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.651633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.651780 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.652161 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.652164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.652166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.652526 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:55.652830 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.653202 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.653385 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":626,"threads_started":1}
W20260926 21:22:55.654915 16374 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52220
W20260926 21:22:55.655835 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16244: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:22:55.656040 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16244
I20260926 21:22:55.673633 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16383
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[15987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457774, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:22:55.692474 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:55.674534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.14.142.3:34555)
0926 21:22:55.674753 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:55.674759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:55.674782 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:22:55.674927 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:55.674933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:55.675013 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:55.675128 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:55.676655 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.677944 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.679628 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:55.680292 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:55.680429 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:55.684733 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:55.684753 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:55.684775 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:55.684844 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:55.689240 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.690105 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.690115 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.690119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.690287 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:55.690669 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:55.690674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:55.690678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:55.691148 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:55.691489 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:55.691822 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 21:22:55.692089 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260926 21:22:55.693233 16514 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45314
W20260926 21:22:55.693997 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16383: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:22:55.694164 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16383
I20260926 21:22:55.716025 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16000
I20260926 21:22:55.717276 16072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:22:55.844921 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16000
2026-09-26T21:22:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5191 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[16623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[16623](info): set up 2 sockets
Sep 26 21:22:55 dist-test-slave-4d5t krb5kdc[16623](info): commencing operation
krb5kdc: starting...
W20260926 21:22:57.009380 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:22:57 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:22:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:22:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:22:57.216439 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38315
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:38315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:57.551292 16636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:57.551625 16636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:57.551800 16636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:57.559999 16636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:22:57.560261 16636 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 21:22:57.560392 16636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:57.560514 16636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:57.560639 16636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:22:57.560751 16636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:22:57.572225 16636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:38315
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:57.573776 16636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:57.575275 16636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:57.588289 16642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:57.588346 16641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:57.588307 16644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:57.589577 16636 server_base.cc:1034] running on GCE node
I20260926 21:22:57.590600 16636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:57.592429 16636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:57.593796 16636 hybrid_clock.cc:648] HybridClock initialized: now 1790457777593721 us; error 45 us; skew 500 ppm
I20260926 21:22:57.594218 16636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:57 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457777, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:57.598440 16636 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:22:57.600881 16636 webserver.cc:460] Webserver started at http://127.14.142.62:38283/ using document root <none> and password file <none>
I20260926 21:22:57.601744 16636 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:57.601907 16636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:57.602267 16636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:57.605811 16636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "362ded96812b46d8adb98e7335ea78d3"
format_stamp: "Formatted at 2026-09-26 21:22:57 on dist-test-slave-4d5t"
server_key: "11681bd1a7c619c39e3b3764432e8c36"
server_key_iv: "b71636d1ad87c942f831ca6c5c17c1a4"
server_key_version: "encryptionkey@0"
I20260926 21:22:57.606760 16636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "362ded96812b46d8adb98e7335ea78d3"
format_stamp: "Formatted at 2026-09-26 21:22:57 on dist-test-slave-4d5t"
server_key: "11681bd1a7c619c39e3b3764432e8c36"
server_key_iv: "b71636d1ad87c942f831ca6c5c17c1a4"
server_key_version: "encryptionkey@0"
I20260926 21:22:57.613083 16636 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:22:57.617815 16651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:57.619549 16636 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:22:57.619831 16636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "362ded96812b46d8adb98e7335ea78d3"
format_stamp: "Formatted at 2026-09-26 21:22:57 on dist-test-slave-4d5t"
server_key: "11681bd1a7c619c39e3b3764432e8c36"
server_key_iv: "b71636d1ad87c942f831ca6c5c17c1a4"
server_key_version: "encryptionkey@0"
I20260926 21:22:57.620141 16636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:57.646199 16636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:57.651039 16636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:57.651338 16636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:57.678115 16636 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:38315
I20260926 21:22:57.678143 16712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:38315 every 8 connection(s)
I20260926 21:22:57.679570 16636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:22:57.683725 16713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:57.688047 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16636
I20260926 21:22:57.688254 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:22:57.689126 14904 external_mini_cluster.cc:1428] Setting key 3b4231fb8dec33e9b4111d4e6904a61c
I20260926 21:22:57.696077 16713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Bootstrap starting.
I20260926 21:22:57.699352 16713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:57.700445 16713 log.cc:824] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:57.703285 16713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: No bootstrap required, opened a new log
Sep 26 21:22:57 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:57.712404 16713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362ded96812b46d8adb98e7335ea78d3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38315 } }
I20260926 21:22:57.713198 16713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:57.713485 16716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:57.691857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.14.142.62:38315)
0926 21:22:57.692830 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:57.692844 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:57.692895 (+ 51us) server_negotiation.cc:407] Connection header received
0926 21:22:57.694698 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:57.694723 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:57.695150 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:57.695567 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:57.697165 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:57.698496 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:57.699867 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:57.700678 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:57.700881 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:57.705226 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:57.705234 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:57.705250 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:57.705295 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:57.708542 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:57.709358 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:57.709374 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:57.709381 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:57.709546 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:57.709932 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:57.709937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:57.709940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:57.710597 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:57.711017 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:57.711513 (+ 496us) server_negotiation.cc:299] Negotiation successful
0926 21:22:57.712025 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":371,"threads_started":1}
I20260926 21:22:57.713443 16713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362ded96812b46d8adb98e7335ea78d3, State: Initialized, Role: FOLLOWER
I20260926 21:22:57.714488 16713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362ded96812b46d8adb98e7335ea78d3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38315 } }
I20260926 21:22:57.714956 16713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:22:57.715159 16713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:22:57.715374 16713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:57.718128 16713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362ded96812b46d8adb98e7335ea78d3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38315 } }
I20260926 21:22:57.718722 16713 leader_election.cc:302] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 362ded96812b46d8adb98e7335ea78d3; no voters:
I20260926 21:22:57.719723 16713 leader_election.cc:288] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:22:57.719780 16718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:57.721318 16718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [term 1 LEADER]: Becoming Leader. State: Replica: 362ded96812b46d8adb98e7335ea78d3, State: Running, Role: LEADER
I20260926 21:22:57.721869 16718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362ded96812b46d8adb98e7335ea78d3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38315 } }
I20260926 21:22:57.722116 16713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:22:57.725525 16719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "362ded96812b46d8adb98e7335ea78d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362ded96812b46d8adb98e7335ea78d3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38315 } } }
I20260926 21:22:57.726095 16719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:57.727705 16725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:22:57.727520 16720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 362ded96812b46d8adb98e7335ea78d3. Latest consensus state: current_term: 1 leader_uuid: "362ded96812b46d8adb98e7335ea78d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362ded96812b46d8adb98e7335ea78d3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38315 } } }
I20260926 21:22:57.727823 16720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3 [sys.catalog]: This master's current role is: LEADER
I20260926 21:22:57.734440 16725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:22:57.745286 16725 catalog_manager.cc:1347] Generated new cluster ID: afe9953975f14574bf8ed3ce47bd8619
I20260926 21:22:57.745518 16725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:22:57.762750 16725 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:22:57.764120 16725 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:22:57.772046 16725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Generated new TSK 0
I20260926 21:22:57.773029 16725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:22:57.833748 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38315
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:58.217412 16739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:58.217716 16739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:58.217892 16739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:58.226202 16739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:58.226354 16739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:58.226554 16739 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 21:22:58.226646 16739 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:22:58.226797 16739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:22:58.238348 16739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:38315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:58.239777 16739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:58.241313 16739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:58.256134 16747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:58.256695 16744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:58.256810 16745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:58.257758 16739 server_base.cc:1034] running on GCE node
I20260926 21:22:58.258462 16739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:58.259473 16739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:58.260707 16739 hybrid_clock.cc:648] HybridClock initialized: now 1790457778260647 us; error 38 us; skew 500 ppm
I20260926 21:22:58.261090 16739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:58 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457778, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:58.264995 16739 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:22:58.266708 16739 webserver.cc:460] Webserver started at http://127.14.142.1:37675/ using document root <none> and password file <none>
I20260926 21:22:58.267477 16739 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:58.267642 16739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:58.267997 16739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:58.271322 16739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "a3058222abd04ed58d85471e75fde451"
format_stamp: "Formatted at 2026-09-26 21:22:58 on dist-test-slave-4d5t"
server_key: "1f4bb4832bc4a95eaab3fbb6a34a1c1c"
server_key_iv: "b6cacdcfe612830a58f3678768d6ec12"
server_key_version: "encryptionkey@0"
I20260926 21:22:58.272298 16739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "a3058222abd04ed58d85471e75fde451"
format_stamp: "Formatted at 2026-09-26 21:22:58 on dist-test-slave-4d5t"
server_key: "1f4bb4832bc4a95eaab3fbb6a34a1c1c"
server_key_iv: "b6cacdcfe612830a58f3678768d6ec12"
server_key_version: "encryptionkey@0"
I20260926 21:22:58.278100 16739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 21:22:58.281643 16754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:58.283087 16739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:22:58.283267 16739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "a3058222abd04ed58d85471e75fde451"
format_stamp: "Formatted at 2026-09-26 21:22:58 on dist-test-slave-4d5t"
server_key: "1f4bb4832bc4a95eaab3fbb6a34a1c1c"
server_key_iv: "b6cacdcfe612830a58f3678768d6ec12"
server_key_version: "encryptionkey@0"
I20260926 21:22:58.283514 16739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:58.300312 16739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:58.304921 16739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:58.305225 16739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:58.308207 16739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:58.308369 16739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:58.308511 16739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:58.308629 16739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 26 21:22:58 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457778, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:58.339238 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.309012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:49723 (local address 127.14.142.62:38315)
0926 21:22:58.309508 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:58.309515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:58.311284 (+ 1769us) server_negotiation.cc:407] Connection header received
0926 21:22:58.314971 (+ 3687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:58.314980 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:58.315099 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:58.315310 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:58.318237 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.319365 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.323724 (+ 4359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.324428 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.324616 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.330374 (+ 5758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:58.330380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:58.330383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:58.330420 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:58.334045 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.335518 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.335525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.335530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.335732 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.336604 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.336609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.336613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.337325 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:58.337554 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:58.338491 (+ 937us) server_negotiation.cc:299] Negotiation successful
0926 21:22:58.338775 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260926 21:22:58.340298 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.309729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.14.142.1:49723)
0926 21:22:58.310957 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:58.311065 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:58.314487 (+ 3422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:58.315668 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:58.315684 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:58.316787 (+ 1103us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:58.317956 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.317994 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.319728 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.319752 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.323259 (+ 3507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.323295 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.324756 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.324768 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.325039 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.325985 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:58.326033 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:58.330051 (+ 4018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:58.334314 (+ 4263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.334331 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.334357 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.335313 (+ 956us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.335921 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.335932 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.335939 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.336409 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.337523 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:58.337533 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:58.338021 (+ 488us) client_negotiation.cc:769] Sending connection context
0926 21:22:58.338344 (+ 323us) client_negotiation.cc:240] Negotiation successful
0926 21:22:58.338963 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":423,"threads_started":1}
I20260926 21:22:58.367682 16739 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:36461
I20260926 21:22:58.367771 16874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:36461 every 8 connection(s)
I20260926 21:22:58.369220 16739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:22:58.371495 16875 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38315
I20260926 21:22:58.371742 16875 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:58.372642 16875 heartbeater.cc:511] Master 127.14.142.62:38315 requested a full tablet report, sending...
I20260926 21:22:58.373061 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.355433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.14.142.1:39739)
0926 21:22:58.356248 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:58.356272 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:58.356575 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:58.357298 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:58.357306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:58.358018 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:58.358359 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.358369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.359924 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.359933 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.362312 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.362333 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.363405 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.363414 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.363677 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.364717 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:58.364744 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:58.366015 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:58.369632 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.369640 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.369646 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.370732 (+ 1086us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.371157 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.371162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.371166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.371342 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.372020 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:58.372023 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:58.372189 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 21:22:58.372306 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 21:22:58.372637 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":497,"threads_started":1}
I20260926 21:22:58.373077 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.355277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:39739 (local address 127.14.142.62:38315)
0926 21:22:58.355495 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:58.355505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:58.356386 (+ 881us) server_negotiation.cc:407] Connection header received
0926 21:22:58.356812 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:58.356817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:58.356890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:58.357003 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:58.358525 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.359704 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.362509 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.363077 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.363263 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.366253 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:58.366290 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:58.366297 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:58.366350 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:58.369494 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.370883 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.370891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.370894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.371038 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.371470 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.371477 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.371482 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.371905 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:58.372068 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:58.372463 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 21:22:58.372733 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 21:22:58.374851 16667 ts_manager.cc:194] Registered new tserver with Master: a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461)
I20260926 21:22:58.376839 16667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:49723
I20260926 21:22:58.378824 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16739
I20260926 21:22:58.379101 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:22:58.380137 14904 external_mini_cluster.cc:1428] Setting key 35619ea901ee83748099d19c89603636
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:22:58.424938 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38315
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:58.767732 16879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:58.768162 16879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:58.768384 16879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:58.777788 16879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:58.777963 16879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:58.778188 16879 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 21:22:58.778297 16879 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:22:58.778470 16879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:22:58.790335 16879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:38315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:58.791706 16879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:58.793156 16879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:58.806433 16884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:58.806525 16887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:58.806583 16885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:58.807258 16879 server_base.cc:1034] running on GCE node
I20260926 21:22:58.807987 16879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:58.809067 16879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:58.810315 16879 hybrid_clock.cc:648] HybridClock initialized: now 1790457778810263 us; error 44 us; skew 500 ppm
I20260926 21:22:58.810664 16879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:58 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457778, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:58.814152 16879 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:22:58.815807 16879 webserver.cc:460] Webserver started at http://127.14.142.2:42647/ using document root <none> and password file <none>
I20260926 21:22:58.816486 16879 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:58.816679 16879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:58.817022 16879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:58.820453 16879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "bacd8b26434a406bacd598359525b51c"
format_stamp: "Formatted at 2026-09-26 21:22:58 on dist-test-slave-4d5t"
server_key: "f91def959eb4e173b647bdacf7ebf386"
server_key_iv: "8ad1b20844fa0d8fd5be6188ad30e6bd"
server_key_version: "encryptionkey@0"
I20260926 21:22:58.821516 16879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "bacd8b26434a406bacd598359525b51c"
format_stamp: "Formatted at 2026-09-26 21:22:58 on dist-test-slave-4d5t"
server_key: "f91def959eb4e173b647bdacf7ebf386"
server_key_iv: "8ad1b20844fa0d8fd5be6188ad30e6bd"
server_key_version: "encryptionkey@0"
I20260926 21:22:58.827754 16879 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 21:22:58.831825 16894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:58.833508 16879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 21:22:58.833709 16879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "bacd8b26434a406bacd598359525b51c"
format_stamp: "Formatted at 2026-09-26 21:22:58 on dist-test-slave-4d5t"
server_key: "f91def959eb4e173b647bdacf7ebf386"
server_key_iv: "8ad1b20844fa0d8fd5be6188ad30e6bd"
server_key_version: "encryptionkey@0"
I20260926 21:22:58.833940 16879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:58.861277 16879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:58.865787 16879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:58.866101 16879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:58.868898 16879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:58.869048 16879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:58.869179 16879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:58.869292 16879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:22:58 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457778, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:58.893694 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.869945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:60897 (local address 127.14.142.62:38315)
0926 21:22:58.870187 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:58.870194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:58.871687 (+ 1493us) server_negotiation.cc:407] Connection header received
0926 21:22:58.874992 (+ 3305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:58.874999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:58.875101 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:58.875247 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:58.877460 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.878334 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.880971 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.881620 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.881759 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.885907 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:58.885916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:58.885925 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:58.885980 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:58.889956 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.890838 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.890844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.890855 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.890983 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.891490 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.891501 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.891503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.891812 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:58.892002 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:58.893024 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0926 21:22:58.893266 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 21:22:58.894802 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.870535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.14.142.2:60897)
0926 21:22:58.871481 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:58.871541 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:58.874562 (+ 3021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:58.875532 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:58.875547 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:58.876409 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:58.877266 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.877284 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.878505 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.878513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.880808 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.880821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.881892 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.881901 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.882112 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.882841 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:58.882870 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:58.885604 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:58.890144 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.890155 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.890176 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.890684 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.891108 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.891114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.891118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.891349 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.891958 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:58.891968 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:58.892506 (+ 538us) client_negotiation.cc:769] Sending connection context
0926 21:22:58.892920 (+ 414us) client_negotiation.cc:240] Negotiation successful
0926 21:22:58.893519 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":379,"threads_started":1}
I20260926 21:22:58.922183 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.903857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:52085 (local address 127.14.142.62:38315)
0926 21:22:58.904132 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:58.904139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:58.904707 (+ 568us) server_negotiation.cc:407] Connection header received
0926 21:22:58.905039 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:58.905045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:58.905157 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:58.905291 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:58.907014 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.907888 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.910539 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.911099 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.911285 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.913693 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:58.913698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:58.913701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:58.913734 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:58.917255 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.918935 (+ 1680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.918940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.918942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.919107 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:58.919954 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:58.919959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:58.919962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:58.920477 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:58.920747 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:58.921492 (+ 745us) server_negotiation.cc:299] Negotiation successful
0926 21:22:58.921709 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 21:22:58.922683 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:58.903764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.14.142.2:52085)
0926 21:22:58.904576 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:58.904602 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:58.904802 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:58.905367 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:58.905376 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:58.906226 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:58.906747 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.906765 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.908043 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.908052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.910317 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:58.910342 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:58.911360 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:58.911367 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:58.911572 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:58.912372 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:58.912390 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:58.913478 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:58.917424 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.917438 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.917445 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.918645 (+ 1200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.919265 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:58.919274 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:58.919281 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:58.919703 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:58.920717 (+ 1014us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:58.920726 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:58.921070 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 21:22:58.921323 (+ 253us) client_negotiation.cc:240] Negotiation successful
0926 21:22:58.921894 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":403,"threads_started":1}
I20260926 21:22:58.934491 16879 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:41869
I20260926 21:22:58.934537 17013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:41869 every 8 connection(s)
I20260926 21:22:58.935886 16879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:22:58.938184 17014 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38315
I20260926 21:22:58.938463 17014 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:58.939234 17014 heartbeater.cc:511] Master 127.14.142.62:38315 requested a full tablet report, sending...
I20260926 21:22:58.940758 16667 ts_manager.cc:194] Registered new tserver with Master: bacd8b26434a406bacd598359525b51c (127.14.142.2:41869)
I20260926 21:22:58.941668 16667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:60897
I20260926 21:22:58.946328 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 16879
I20260926 21:22:58.946563 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:22:58.947074 14904 external_mini_cluster.cc:1428] Setting key d337c5bfb49ecb599c6d9786ddc1d9ac
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:22:58.988430 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38315
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:22:59.345167 17018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:22:59.345470 17018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:22:59.345573 17018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:22:59.353036 17018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:22:59.353157 17018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:22:59.353245 17018 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 21:22:59.353297 17018 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:22:59.353367 17018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:22:59.363417 17018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:38315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:22:59.364749 17018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:22:59.366004 17018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:22:59.379092 17023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:59.379200 17026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:22:59.379251 17024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:22:59.379717 16875 heartbeater.cc:503] Master 127.14.142.62:38315 was elected leader, sending a full tablet report...
I20260926 21:22:59.380604 17018 server_base.cc:1034] running on GCE node
I20260926 21:22:59.381426 17018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:22:59.382512 17018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:22:59.383785 17018 hybrid_clock.cc:648] HybridClock initialized: now 1790457779383710 us; error 58 us; skew 500 ppm
I20260926 21:22:59.384173 17018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:22:59 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457779, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:22:59.393503 17018 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:22:59.395130 17018 webserver.cc:460] Webserver started at http://127.14.142.3:36893/ using document root <none> and password file <none>
I20260926 21:22:59.395968 17018 fs_manager.cc:359] Metadata directory not provided
I20260926 21:22:59.396073 17018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:22:59.396340 17018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:22:59.399704 17018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "026353374fa64cafbcbc7fafaee5c724"
format_stamp: "Formatted at 2026-09-26 21:22:59 on dist-test-slave-4d5t"
server_key: "1627f4e61d4809d3bbd472f3266d55b2"
server_key_iv: "fc4095aaf5aa24daf9d241919a38668e"
server_key_version: "encryptionkey@0"
I20260926 21:22:59.400482 17018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "026353374fa64cafbcbc7fafaee5c724"
format_stamp: "Formatted at 2026-09-26 21:22:59 on dist-test-slave-4d5t"
server_key: "1627f4e61d4809d3bbd472f3266d55b2"
server_key_iv: "fc4095aaf5aa24daf9d241919a38668e"
server_key_version: "encryptionkey@0"
I20260926 21:22:59.406229 17018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 21:22:59.411043 17033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:22:59.412700 17018 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 21:22:59.412904 17018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "026353374fa64cafbcbc7fafaee5c724"
format_stamp: "Formatted at 2026-09-26 21:22:59 on dist-test-slave-4d5t"
server_key: "1627f4e61d4809d3bbd472f3266d55b2"
server_key_iv: "fc4095aaf5aa24daf9d241919a38668e"
server_key_version: "encryptionkey@0"
I20260926 21:22:59.413163 17018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:22:59.434935 17018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:22:59.440814 17018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:22:59.441296 17018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:22:59.444640 17018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:22:59.444756 17018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 21:22:59.444970 17018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:22:59.445077 17018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:22:59 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457779, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:59.475850 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.445145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:33931 (local address 127.14.142.62:38315)
0926 21:22:59.445979 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.445986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.447161 (+ 1175us) server_negotiation.cc:407] Connection header received
0926 21:22:59.450533 (+ 3372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.450540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.450638 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.450765 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:59.453408 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.454537 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.457474 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.458268 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.458469 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.466142 (+ 7673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:59.466149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:59.466153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:59.466202 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:59.471896 (+ 5694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.473007 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.473014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.473018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.473179 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.473716 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.473722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.473725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.474055 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:59.474268 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.475126 (+ 858us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.475402 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":604,"threads_started":1}
I20260926 21:22:59.477587 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.445900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.14.142.3:33931)
0926 21:22:59.446931 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.446993 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.450095 (+ 3102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.451232 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.451250 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.452072 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:59.453179 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.453216 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.454709 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.454722 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.457188 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.457207 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.458694 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.458705 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.459043 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.460247 (+ 1204us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:59.460279 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:59.465616 (+ 5337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:59.472108 (+ 6492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:59.472121 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:59.472155 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:59.472824 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:59.473321 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:59.473326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:59.473331 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:59.473572 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:59.474223 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:59.474233 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:59.474681 (+ 448us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.475000 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.475644 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":396,"threads_started":1}
I20260926 21:22:59.503810 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.484900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:44515 (local address 127.14.142.62:38315)
0926 21:22:59.485140 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.485146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.486286 (+ 1140us) server_negotiation.cc:407] Connection header received
0926 21:22:59.486587 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.486591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.486660 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.486988 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:59.488870 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.489759 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.492258 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.493027 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.493179 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.495753 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:59.495763 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:59.495771 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:59.495823 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:59.500112 (+ 4289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.501241 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.501246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.501249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.501408 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.501849 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.501857 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.501862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.502370 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:59.502590 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.503092 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.503404 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260926 21:22:59.503935 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.485173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.14.142.3:44515)
0926 21:22:59.486134 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.486163 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.486412 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.487084 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.487090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.487949 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:59.488684 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.488705 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.489988 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.489995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.492082 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.492104 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.493236 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.493243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.493405 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.494132 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:59.494154 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:59.495444 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:59.500314 (+ 4870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:59.500322 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:59.500329 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:59.501076 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:59.501556 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:59.501560 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:59.501563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:59.501707 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:59.502524 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:59.502528 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:59.502742 (+ 214us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.502974 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.503413 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":451,"threads_started":1}
I20260926 21:22:59.505678 17018 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:43903
I20260926 21:22:59.505693 17153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:43903 every 8 connection(s)
I20260926 21:22:59.507019 17018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:22:59.509776 17154 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38315
I20260926 21:22:59.510040 17154 heartbeater.cc:464] Registering TS with master...
I20260926 21:22:59.510782 17154 heartbeater.cc:511] Master 127.14.142.62:38315 requested a full tablet report, sending...
I20260926 21:22:59.511951 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17018
I20260926 21:22:59.512166 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:22:59.512825 14904 external_mini_cluster.cc:1428] Setting key 3c0ddecc376223f991fe58d90c477f98
I20260926 21:22:59.513098 16667 ts_manager.cc:194] Registered new tserver with Master: 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:22:59.514132 16667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:33931
I20260926 21:22:59.516678 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:22:59 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 21:22:59 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:59.570647 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.554430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.14.142.62:38315)
0926 21:22:59.554707 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.554715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.554919 (+ 204us) server_negotiation.cc:407] Connection header received
0926 21:22:59.555406 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.555411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.555480 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.555602 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:59.557103 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.557842 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.559483 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.559960 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.560156 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.564815 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:59.564821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:59.564825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:59.564886 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:59.567598 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.568378 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.568385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.568388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.568542 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.569028 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.569034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.569038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.569368 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:59.569660 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.569982 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.570240 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 21:22:59.575595 16667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:22:59.583263 16667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:22:59.617712 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.598077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.14.142.3:43903)
0926 21:22:59.607528 (+ 9451us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.607544 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.608790 (+ 1246us) server_negotiation.cc:407] Connection header received
0926 21:22:59.609311 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.609324 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.609668 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.609873 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:59.611308 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.613088 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.615128 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.616402 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.616562 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.616841 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.617152 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.617352 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9174,"thread_start_us":9481,"threads_started":1}
I20260926 21:22:59.617859 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.597802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43903 (local address 127.0.0.1:52844)
0926 21:22:59.608519 (+ 10717us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.608635 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.609011 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.610327 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.610340 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.610390 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:59.611034 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.611054 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.613245 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.613252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.614928 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.614946 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.616621 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.616629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.616881 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.616933 (+ 52us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.617106 (+ 173us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.617447 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10377,"mutex_wait_us":47,"thread_start_us":13443,"threads_started":1}
I20260926 21:22:59.620087 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.597975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36461 (local address 127.0.0.1:54250)
0926 21:22:59.611246 (+ 13271us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.611271 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.611408 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.612118 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.612121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.612143 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:59.612533 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.612600 (+ 67us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.614560 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.614565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.617077 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.617092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.619214 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.619221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.619392 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.619429 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.619528 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.619673 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13065,"mutex_wait_us":21,"thread_start_us":12881,"threads_started":1}
I20260926 21:22:59.620430 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.598677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55528 (local address 127.14.142.2:41869)
0926 21:22:59.610625 (+ 11948us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.610637 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.611669 (+ 1032us) server_negotiation.cc:407] Connection header received
0926 21:22:59.611961 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.611968 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.612174 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.612337 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:59.613170 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.614591 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.617700 (+ 3109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.619003 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.619113 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.619310 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.619938 (+ 628us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.620121 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11777,"thread_start_us":11650,"threads_started":1}
I20260926 21:22:59.620836 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.599717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54250 (local address 127.14.142.1:36461)
0926 21:22:59.610207 (+ 10490us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.610216 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.611543 (+ 1327us) server_negotiation.cc:407] Connection header received
0926 21:22:59.611645 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.611651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.611909 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.612775 (+ 866us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:59.612940 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.614400 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.617252 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.618947 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.619134 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.619313 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.620354 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.620526 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10299,"thread_start_us":10109,"threads_started":1}
I20260926 21:22:59.621178 17167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.598456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41869 (local address 127.0.0.1:55528)
0926 21:22:59.611553 (+ 13097us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.611574 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.611774 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.612595 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.612600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.612623 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:59.612973 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.612982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.614763 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.614773 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.617496 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.617509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.619521 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.619528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.619703 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.619739 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.620721 (+ 982us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.620832 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12962,"mutex_wait_us":24,"thread_start_us":9586,"threads_started":1}
I20260926 21:22:59.622292 17084 tablet_service.cc:1467] Processing CreateTablet for tablet f394cb9145eb44479a845a8fa14a6407 (DEFAULT_TABLE table=test-table [id=2a4695be71d444ce9d7cb6c3cec31724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:59.623936 17084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f394cb9145eb44479a845a8fa14a6407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:59.624598 16805 tablet_service.cc:1467] Processing CreateTablet for tablet f394cb9145eb44479a845a8fa14a6407 (DEFAULT_TABLE table=test-table [id=2a4695be71d444ce9d7cb6c3cec31724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:59.624643 16944 tablet_service.cc:1467] Processing CreateTablet for tablet f394cb9145eb44479a845a8fa14a6407 (DEFAULT_TABLE table=test-table [id=2a4695be71d444ce9d7cb6c3cec31724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:22:59.625900 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f394cb9145eb44479a845a8fa14a6407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:59.625927 16944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f394cb9145eb44479a845a8fa14a6407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:59.638403 17173 tablet_bootstrap.cc:492] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Bootstrap starting.
I20260926 21:22:59.642581 17173 tablet_bootstrap.cc:654] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:59.643886 17173 log.cc:824] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:59.645260 17174 tablet_bootstrap.cc:492] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Bootstrap starting.
I20260926 21:22:59.646907 17173 tablet_bootstrap.cc:492] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: No bootstrap required, opened a new log
I20260926 21:22:59.647192 17173 ts_tablet_manager.cc:1397] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260926 21:22:59.649169 17174 tablet_bootstrap.cc:654] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:59.650692 17174 log.cc:824] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:59.652685 17175 tablet_bootstrap.cc:492] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Bootstrap starting.
I20260926 21:22:59.654996 17174 tablet_bootstrap.cc:492] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: No bootstrap required, opened a new log
I20260926 21:22:59.655298 17174 ts_tablet_manager.cc:1397] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 21:22:59.656051 17173 raft_consensus.cc:348] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.656522 17175 tablet_bootstrap.cc:654] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:59.656519 17173 raft_consensus.cc:374] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:59.656661 17173 raft_consensus.cc:729] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3058222abd04ed58d85471e75fde451, State: Initialized, Role: FOLLOWER
I20260926 21:22:59.657917 17175 log.cc:824] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Log is configured to *not* fsync() on all Append() calls
I20260926 21:22:59.657444 17173 consensus_queue.cc:260] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.660895 17173 ts_tablet_manager.cc:1428] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260926 21:22:59.661865 17175 tablet_bootstrap.cc:492] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: No bootstrap required, opened a new log
I20260926 21:22:59.662156 17175 ts_tablet_manager.cc:1397] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.000s
I20260926 21:22:59.666069 17174 raft_consensus.cc:348] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.666572 17174 raft_consensus.cc:374] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:59.666671 17174 raft_consensus.cc:729] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Initialized, Role: FOLLOWER
I20260926 21:22:59.667346 17174 consensus_queue.cc:260] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.668617 17154 heartbeater.cc:503] Master 127.14.142.62:38315 was elected leader, sending a full tablet report...
I20260926 21:22:59.669350 17174 ts_tablet_manager.cc:1428] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260926 21:22:59.672967 17175 raft_consensus.cc:348] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.673379 17175 raft_consensus.cc:374] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:59.673489 17175 raft_consensus.cc:729] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacd8b26434a406bacd598359525b51c, State: Initialized, Role: FOLLOWER
I20260926 21:22:59.674167 17175 consensus_queue.cc:260] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.675315 17014 heartbeater.cc:503] Master 127.14.142.62:38315 was elected leader, sending a full tablet report...
I20260926 21:22:59.675778 17175 ts_tablet_manager.cc:1428] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260926 21:22:59.688231 17015 tablet.cc:2369] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:22:59.759320 17155 tablet.cc:2369] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:22:59.825263 17181 raft_consensus.cc:482] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:22:59.825654 17181 raft_consensus.cc:504] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.827358 17181 leader_election.cc:288] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903), a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461)
I20260926 21:22:59.837486 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.827858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:36505 (local address 127.14.142.1:36461)
0926 21:22:59.828079 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.828088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.828757 (+ 669us) server_negotiation.cc:407] Connection header received
0926 21:22:59.829159 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.829165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.829271 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.829363 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:59.830178 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.831519 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.834698 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.836120 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.836328 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.836413 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.837086 (+ 673us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.837227 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:22:59.837486 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.827948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:32829 (local address 127.14.142.3:43903)
0926 21:22:59.828269 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.828275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.829089 (+ 814us) server_negotiation.cc:407] Connection header received
0926 21:22:59.829168 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.829172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.829252 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.829363 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:22:59.830178 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.831519 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.834698 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.836122 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.836333 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.836413 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.837086 (+ 673us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.837227 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260926 21:22:59.837670 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.827830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43903 (local address 127.14.142.2:32829)
0926 21:22:59.828884 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.828912 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.829060 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.829425 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.829431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.829460 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:59.829976 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.829994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.831697 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.831707 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.834443 (+ 2736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.834460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.836444 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.836450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.836785 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.836918 (+ 133us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.837022 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.837190 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":249,"thread_start_us":579,"threads_started":1}
I20260926 21:22:59.838116 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.827815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36461 (local address 127.14.142.2:36505)
0926 21:22:59.828613 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.828650 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.828957 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.829425 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.829431 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.829525 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:22:59.830003 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.830012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.831699 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.831705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.834443 (+ 2738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.834460 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.836443 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.836450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.836785 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.836919 (+ 134us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.837022 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.837223 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":277,"spinlock_wait_cycles":25472,"thread_start_us":327,"threads_started":1}
I20260926 21:22:59.838788 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f394cb9145eb44479a845a8fa14a6407" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724" is_pre_election: true
I20260926 21:22:59.839368 17108 raft_consensus.cc:2454] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:22:59.839354 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f394cb9145eb44479a845a8fa14a6407" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451" is_pre_election: true
I20260926 21:22:59.840121 16895 leader_election.cc:302] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:22:59.840127 16825 raft_consensus.cc:2454] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:22:59.840520 17181 raft_consensus.cc:2781] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:22:59.840716 17181 raft_consensus.cc:482] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:22:59.840866 17181 raft_consensus.cc:3037] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:59.843137 17181 raft_consensus.cc:504] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.843874 17181 leader_election.cc:288] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Requested vote from peers 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903), a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461)
I20260926 21:22:59.844332 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f394cb9145eb44479a845a8fa14a6407" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724"
I20260926 21:22:59.844597 17108 raft_consensus.cc:3037] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:59.844543 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f394cb9145eb44479a845a8fa14a6407" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451"
I20260926 21:22:59.844846 16825 raft_consensus.cc:3037] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:22:59.846863 17108 raft_consensus.cc:2454] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 1.
I20260926 21:22:59.847136 16825 raft_consensus.cc:2454] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 1.
I20260926 21:22:59.847384 16895 leader_election.cc:302] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:22:59.847689 17181 raft_consensus.cc:2781] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:22:59.848220 17181 raft_consensus.cc:686] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 1 LEADER]: Becoming Leader. State: Replica: bacd8b26434a406bacd598359525b51c, State: Running, Role: LEADER
I20260926 21:22:59.848791 17181 consensus_queue.cc:237] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } }
I20260926 21:22:59.854856 16667 catalog_manager.cc:5495] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c reported cstate change: term changed from 0 to 1, leader changed from <none> to bacd8b26434a406bacd598359525b51c (127.14.142.2). New cstate: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } health_report { overall_health: HEALTHY } } }
W20260926 21:22:59.872063 16876 tablet.cc:2369] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 26 21:22:59 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457777, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:22:59.958956 17042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.943140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.14.142.62:38315)
0926 21:22:59.943347 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 21:22:59.943354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:22:59.944007 (+ 653us) server_negotiation.cc:407] Connection header received
0926 21:22:59.944186 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:22:59.944191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:22:59.944281 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:22:59.944416 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:22:59.945588 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.946571 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.947889 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.948346 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.948459 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.953109 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:22:59.953115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:22:59.953118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:22:59.953151 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:22:59.955990 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.956871 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.956876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.956880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.957009 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:22:59.957427 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:22:59.957431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:22:59.957433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:22:59.957709 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:22:59.957867 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 21:22:59.958278 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 21:22:59.958615 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":59}
I20260926 21:22:59.958999 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:22:59.943201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38315 (local address 127.0.0.1:42490)
0926 21:22:59.943847 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0926 21:22:59.943869 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:22:59.944064 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:22:59.944454 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:22:59.944457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:22:59.945143 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:22:59.945431 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.945440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.946733 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.946737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.947734 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:22:59.947745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:22:59.948777 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:22:59.948785 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:22:59.949017 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:22:59.950177 (+ 1160us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:22:59.950198 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:22:59.952899 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:22:59.956154 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:59.956166 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:59.956173 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:59.956733 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:59.957153 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:22:59.957159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:22:59.957167 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:22:59.957291 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:22:59.957841 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:22:59.957850 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:22:59.958028 (+ 178us) client_negotiation.cc:769] Sending connection context
0926 21:22:59.958192 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 21:22:59.958492 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":367,"threads_started":1}
I20260926 21:22:59.964275 16667 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:42490:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 21:22:59.965391 16667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:22:59.973935 16805 tablet_service.cc:1467] Processing CreateTablet for tablet 21d9e974e94a4ef68188e4ccfb213673 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f1d25d5f1bbb48988541173c5355a909]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:22:59.974084 16944 tablet_service.cc:1467] Processing CreateTablet for tablet 21d9e974e94a4ef68188e4ccfb213673 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f1d25d5f1bbb48988541173c5355a909]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:22:59.974280 17084 tablet_service.cc:1467] Processing CreateTablet for tablet 21d9e974e94a4ef68188e4ccfb213673 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f1d25d5f1bbb48988541173c5355a909]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:22:59.975109 17084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d9e974e94a4ef68188e4ccfb213673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:59.975109 16944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d9e974e94a4ef68188e4ccfb213673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:59.975128 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d9e974e94a4ef68188e4ccfb213673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:22:59.982406 17173 tablet_bootstrap.cc:492] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: Bootstrap starting.
I20260926 21:22:59.983978 17174 tablet_bootstrap.cc:492] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: Bootstrap starting.
I20260926 21:22:59.984270 17175 tablet_bootstrap.cc:492] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: Bootstrap starting.
I20260926 21:22:59.984390 17173 tablet_bootstrap.cc:654] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:59.985678 17175 tablet_bootstrap.cc:654] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:59.986094 17174 tablet_bootstrap.cc:654] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: Neither blocks nor log segments found. Creating new log.
I20260926 21:22:59.987594 17173 tablet_bootstrap.cc:492] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: No bootstrap required, opened a new log
I20260926 21:22:59.987761 17173 ts_tablet_manager.cc:1397] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260926 21:22:59.988843 17175 tablet_bootstrap.cc:492] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: No bootstrap required, opened a new log
I20260926 21:22:59.988889 17174 tablet_bootstrap.cc:492] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: No bootstrap required, opened a new log
I20260926 21:22:59.988696 17173 raft_consensus.cc:348] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:22:59.988986 17173 raft_consensus.cc:374] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:59.989001 17175 ts_tablet_manager.cc:1397] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260926 21:22:59.989046 17174 ts_tablet_manager.cc:1397] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 21:22:59.989099 17173 raft_consensus.cc:729] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3058222abd04ed58d85471e75fde451, State: Initialized, Role: FOLLOWER
I20260926 21:22:59.989362 17173 consensus_queue.cc:260] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:22:59.989753 17175 raft_consensus.cc:348] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:22:59.989956 17175 raft_consensus.cc:374] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:59.990000 17173 ts_tablet_manager.cc:1428] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 21:22:59.990082 17175 raft_consensus.cc:729] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacd8b26434a406bacd598359525b51c, State: Initialized, Role: FOLLOWER
I20260926 21:22:59.989893 17174 raft_consensus.cc:348] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:22:59.990180 17174 raft_consensus.cc:374] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:22:59.990373 17174 raft_consensus.cc:729] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Initialized, Role: FOLLOWER
I20260926 21:22:59.990298 17175 consensus_queue.cc:260] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:22:59.990814 17181 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:22:59.991171 17181 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:59.991110 17178 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:22:59.991321 17174 consensus_queue.cc:260] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:22:59.991519 17178 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:22:59.992060 17174 ts_tablet_manager.cc:1428] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 21:22:59.991966 17180 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:22:59.992305 17175 ts_tablet_manager.cc:1428] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 21:22:59.992329 17180 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:00.171479 17181 raft_consensus.cc:482] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:00.171691 17181 raft_consensus.cc:504] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.172276 17181 leader_election.cc:288] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:23:00.172850 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d9e974e94a4ef68188e4ccfb213673" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451" is_pre_election: true
I20260926 21:23:00.172879 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d9e974e94a4ef68188e4ccfb213673" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724" is_pre_election: true
I20260926 21:23:00.173111 16825 raft_consensus.cc:2454] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:23:00.173159 17108 raft_consensus.cc:2454] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:23:00.173578 16897 leader_election.cc:302] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3058222abd04ed58d85471e75fde451, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:00.173885 17181 raft_consensus.cc:2781] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:00.173985 17181 raft_consensus.cc:482] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:00.174103 17181 raft_consensus.cc:3037] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:00.176151 17181 raft_consensus.cc:504] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.176864 17181 leader_election.cc:288] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Requested vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:23:00.177386 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d9e974e94a4ef68188e4ccfb213673" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451"
I20260926 21:23:00.177394 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d9e974e94a4ef68188e4ccfb213673" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724"
I20260926 21:23:00.177639 16825 raft_consensus.cc:3037] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:00.177654 17108 raft_consensus.cc:3037] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:00.179777 17108 raft_consensus.cc:2454] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 1.
I20260926 21:23:00.180189 16825 raft_consensus.cc:2454] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 1.
I20260926 21:23:00.180298 16895 leader_election.cc:302] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:00.180740 17181 raft_consensus.cc:2781] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:00.180939 17181 raft_consensus.cc:686] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [term 1 LEADER]: Becoming Leader. State: Replica: bacd8b26434a406bacd598359525b51c, State: Running, Role: LEADER
I20260926 21:23:00.181435 17181 consensus_queue.cc:237] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.183903 17184 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: TxnStatusTablet state changed. Reason: New leader bacd8b26434a406bacd598359525b51c. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.184230 17184 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:23:00.184944 17196 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 21:23:00.185231 17196 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 21:23:00.185281 16668 catalog_manager.cc:5495] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c reported cstate change: term changed from 0 to 1, leader changed from <none> to bacd8b26434a406bacd598359525b51c (127.14.142.2). New cstate: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:00.233914 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.226931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55540 (local address 127.14.142.2:41869)
0926 21:23:00.227560 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.227565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.227589 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:23:00.227686 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.227689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.227774 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.227923 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:00.228371 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.229321 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.231248 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.231751 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.231855 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.232499 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.233246 (+ 747us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.233377 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.233515 (+ 138us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.233593 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":341,"threads_started":1}
I20260926 21:23:00.233994 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.226749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41869 (local address 127.0.0.1:55540)
0926 21:23:00.226996 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:00.227017 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:00.227170 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:00.227975 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:00.227980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:00.227994 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:23:00.228205 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.228216 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.229461 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.229467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.231082 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.231097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.231927 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.231935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.232096 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.232177 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.233367 (+ 1190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.233389 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 21:23:00.233488 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 21:23:00.233659 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 21:23:00.242075 17108 raft_consensus.cc:1261] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Refusing update from remote peer bacd8b26434a406bacd598359525b51c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:00.242075 16825 raft_consensus.cc:1261] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Refusing update from remote peer bacd8b26434a406bacd598359525b51c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:00.243080 17184 consensus_queue.cc:1035] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [LEADER]: Connected to new peer: Peer: permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:00.243453 17181 consensus_queue.cc:1035] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:00.258841 17180 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: TxnStatusTablet state changed. Reason: New leader bacd8b26434a406bacd598359525b51c. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.259143 17180 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:00.261183 17200 mvcc.cc:205] Tried to move back new op lower bound from 7333715067861168128 to 7333715067628064768. Current Snapshot: MvccSnapshot[applied={T|T < 7333715067861168128}]
I20260926 21:23:00.262100 17184 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.262392 17184 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:23:00.268685 17178 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: TxnStatusTablet state changed. Reason: New leader bacd8b26434a406bacd598359525b51c. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.269022 17178 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:00.273624 17181 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.273995 17181 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P bacd8b26434a406bacd598359525b51c: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:23:00.275772 17203 mvcc.cc:205] Tried to move back new op lower bound from 7333715067861168128 to 7333715067628064768. Current Snapshot: MvccSnapshot[applied={T|T < 7333715067861168128}]
I20260926 21:23:00.287801 17178 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.288120 17178 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P a3058222abd04ed58d85471e75fde451: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:00.289752 17180 tablet_replica.cc:439] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } } }
I20260926 21:23:00.290059 17180 tablet_replica.cc:442] T 21d9e974e94a4ef68188e4ccfb213673 P 026353374fa64cafbcbc7fafaee5c724: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:00.295823 17184 consensus_queue.cc:1035] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:00.299332 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.287636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.14.142.2:41869)
0926 21:23:00.287933 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.287943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.292479 (+ 4536us) server_negotiation.cc:407] Connection header received
0926 21:23:00.292781 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.292789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.292896 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.293011 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:00.293490 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.294770 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.296869 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.297491 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.297661 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.298300 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.298472 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.298560 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.298901 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.299002 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 21:23:00.303746 17181 consensus_queue.cc:1035] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [LEADER]: Connected to new peer: Peer: permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:00.321419 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.314478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:47249 (local address 127.14.142.2:41869)
0926 21:23:00.314735 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.314743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.314770 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:00.314968 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.314974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.315071 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.315199 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:00.315788 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.317042 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.318770 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.319497 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.319651 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.320424 (+ 773us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.320602 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.320717 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.320871 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.320960 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 21:23:00.321425 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.314027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41869 (local address 127.14.142.2:47249)
0926 21:23:00.314676 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:00.314696 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:00.314836 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:00.315233 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:00.315237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:00.315248 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:23:00.315612 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.315620 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.317191 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.317195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.318623 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.318637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.319821 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.319830 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.319994 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.320139 (+ 145us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.320716 (+ 577us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.320726 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 21:23:00.320843 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 21:23:00.320998 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":496,"threads_started":1}
I20260926 21:23:00.414127 16665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:00.414439 16665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:00.418565 16665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Sending DeleteTablet for 3 replicas of tablet f394cb9145eb44479a845a8fa14a6407
I20260926 21:23:00.419869 16805 tablet_service.cc:1514] Processing DeleteTablet for tablet f394cb9145eb44479a845a8fa14a6407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:00 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:54250
I20260926 21:23:00.419868 17084 tablet_service.cc:1514] Processing DeleteTablet for tablet f394cb9145eb44479a845a8fa14a6407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:00 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:52844
I20260926 21:23:00.420815 17218 tablet_replica.cc:330] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: stopping tablet replica
I20260926 21:23:00.420665 16943 tablet_service.cc:1514] Processing DeleteTablet for tablet f394cb9145eb44479a845a8fa14a6407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:00 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55528
I20260926 21:23:00.421329 17218 raft_consensus.cc:2229] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:00.421784 17218 raft_consensus.cc:2258] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:00.423954 17218 ts_tablet_manager.cc:1905] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:00.423729 16665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:00.425438 16665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:00.432229 17218 ts_tablet_manager.cc:1918] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:00.432531 17218 log.cc:1196] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/f394cb9145eb44479a845a8fa14a6407
I20260926 21:23:00.433176 17218 ts_tablet_manager.cc:1939] T f394cb9145eb44479a845a8fa14a6407 P 026353374fa64cafbcbc7fafaee5c724: Deleting consensus metadata
I20260926 21:23:00.435237 16652 catalog_manager.cc:4841] TS 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903): tablet f394cb9145eb44479a845a8fa14a6407 (table test-table [id=2a4695be71d444ce9d7cb6c3cec31724]) successfully deleted
I20260926 21:23:00.435401 17219 tablet_replica.cc:330] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: stopping tablet replica
I20260926 21:23:00.435920 17219 raft_consensus.cc:2229] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:00.436331 17219 raft_consensus.cc:2258] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:00.436869 16804 tablet_service.cc:1467] Processing CreateTablet for tablet ad10bf2665dc4776916065a37401e1d7 (DEFAULT_TABLE table=test-table [id=1b0b1a32e9f04e3191f36f857d33aaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:00.437364 16804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad10bf2665dc4776916065a37401e1d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:00.441685 16944 tablet_service.cc:1467] Processing CreateTablet for tablet ad10bf2665dc4776916065a37401e1d7 (DEFAULT_TABLE table=test-table [id=1b0b1a32e9f04e3191f36f857d33aaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:00.442220 16944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad10bf2665dc4776916065a37401e1d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:00.445528 17084 tablet_service.cc:1467] Processing CreateTablet for tablet ad10bf2665dc4776916065a37401e1d7 (DEFAULT_TABLE table=test-table [id=1b0b1a32e9f04e3191f36f857d33aaad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:00.445883 17220 tablet_replica.cc:330] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: stopping tablet replica
I20260926 21:23:00.446038 17084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad10bf2665dc4776916065a37401e1d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:00.446405 17220 raft_consensus.cc:2229] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:00.447171 17220 raft_consensus.cc:2258] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:00.449586 17220 ts_tablet_manager.cc:1905] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:00.453953 17174 tablet_bootstrap.cc:492] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Bootstrap starting.
I20260926 21:23:00.453997 17173 tablet_bootstrap.cc:492] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Bootstrap starting.
I20260926 21:23:00.455714 17174 tablet_bootstrap.cc:654] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:00.455704 17173 tablet_bootstrap.cc:654] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:00.458022 17220 ts_tablet_manager.cc:1918] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:00.458228 17220 log.cc:1196] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/f394cb9145eb44479a845a8fa14a6407
I20260926 21:23:00.458961 17220 ts_tablet_manager.cc:1939] T f394cb9145eb44479a845a8fa14a6407 P bacd8b26434a406bacd598359525b51c: Deleting consensus metadata
I20260926 21:23:00.459939 17175 tablet_bootstrap.cc:492] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Bootstrap starting.
I20260926 21:23:00.461784 17175 tablet_bootstrap.cc:654] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:00.461792 16655 catalog_manager.cc:4841] TS bacd8b26434a406bacd598359525b51c (127.14.142.2:41869): tablet f394cb9145eb44479a845a8fa14a6407 (table test-table [id=2a4695be71d444ce9d7cb6c3cec31724]) successfully deleted
I20260926 21:23:00.464852 17175 tablet_bootstrap.cc:492] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: No bootstrap required, opened a new log
I20260926 21:23:00.465116 17175 ts_tablet_manager.cc:1397] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260926 21:23:00.466506 17219 ts_tablet_manager.cc:1905] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:00.466382 17175 raft_consensus.cc:348] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.468691 17175 raft_consensus.cc:374] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:00.468847 17175 raft_consensus.cc:729] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacd8b26434a406bacd598359525b51c, State: Initialized, Role: FOLLOWER
I20260926 21:23:00.469251 17175 consensus_queue.cc:260] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.470127 17175 ts_tablet_manager.cc:1428] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 21:23:00.473225 17174 tablet_bootstrap.cc:492] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: No bootstrap required, opened a new log
I20260926 21:23:00.474936 17174 ts_tablet_manager.cc:1397] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.006s
I20260926 21:23:00.474949 17173 tablet_bootstrap.cc:492] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: No bootstrap required, opened a new log
I20260926 21:23:00.475108 17173 ts_tablet_manager.cc:1397] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.001s
I20260926 21:23:00.475668 17174 raft_consensus.cc:348] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.475914 17174 raft_consensus.cc:374] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:00.475847 17173 raft_consensus.cc:348] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.476017 17174 raft_consensus.cc:729] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Initialized, Role: FOLLOWER
I20260926 21:23:00.476089 17173 raft_consensus.cc:374] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:00.476159 17173 raft_consensus.cc:729] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3058222abd04ed58d85471e75fde451, State: Initialized, Role: FOLLOWER
I20260926 21:23:00.476313 17174 consensus_queue.cc:260] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.476383 17173 consensus_queue.cc:260] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.476982 17173 ts_tablet_manager.cc:1428] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 21:23:00.479689 17174 ts_tablet_manager.cc:1428] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260926 21:23:00.481166 17219 ts_tablet_manager.cc:1918] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:00.482057 17219 log.cc:1196] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/f394cb9145eb44479a845a8fa14a6407
I20260926 21:23:00.482586 17219 ts_tablet_manager.cc:1939] T f394cb9145eb44479a845a8fa14a6407 P a3058222abd04ed58d85471e75fde451: Deleting consensus metadata
I20260926 21:23:00.484539 16654 catalog_manager.cc:4841] TS a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461): tablet f394cb9145eb44479a845a8fa14a6407 (table test-table [id=2a4695be71d444ce9d7cb6c3cec31724]) successfully deleted
I20260926 21:23:00.641573 17180 raft_consensus.cc:482] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:00.641891 17180 raft_consensus.cc:504] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.644304 17180 leader_election.cc:288] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), bacd8b26434a406bacd598359525b51c (127.14.142.2:41869)
I20260926 21:23:00.657797 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.646094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:41977 (local address 127.14.142.2:41869)
0926 21:23:00.646365 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.646373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.652012 (+ 5639us) server_negotiation.cc:407] Connection header received
0926 21:23:00.652341 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.652347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.652440 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.652532 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:00.653044 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.653803 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.655603 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.656529 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.656768 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.656855 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.657310 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.657482 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260926 21:23:00.658805 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.645531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41869 (local address 127.14.142.3:41977)
0926 21:23:00.651873 (+ 6342us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:00.651897 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:00.652145 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:00.652568 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:00.652572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:00.652594 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:00.652896 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.652904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.653947 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.653955 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.655445 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.655457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.656855 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.656862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.657066 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.657105 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 21:23:00.658122 (+ 1017us) client_negotiation.cc:240] Negotiation successful
0926 21:23:00.658312 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6149,"mutex_wait_us":5,"thread_start_us":6049,"threads_started":1}
I20260926 21:23:00.659242 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.645142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:50831 (local address 127.14.142.1:36461)
0926 21:23:00.651450 (+ 6308us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.651456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.651484 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:23:00.651675 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.651681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.651760 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.651868 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:00.652612 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.653874 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.656728 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.657873 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.658061 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.658123 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.658572 (+ 449us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.658731 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6113,"thread_start_us":6015,"threads_started":1}
I20260926 21:23:00.659845 16969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad10bf2665dc4776916065a37401e1d7" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bacd8b26434a406bacd598359525b51c" is_pre_election: true
I20260926 21:23:00.660300 16969 raft_consensus.cc:2454] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 0.
I20260926 21:23:00.660923 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.644680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36461 (local address 127.14.142.3:50831)
0926 21:23:00.651297 (+ 6617us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:00.651322 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:00.651501 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:00.651959 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:00.651967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:00.651995 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:00.652335 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.652347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.654081 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.654091 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.656496 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.656513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.658281 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.658286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.658400 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.658423 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 21:23:00.658532 (+ 109us) client_negotiation.cc:240] Negotiation successful
0926 21:23:00.658663 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6476,"thread_start_us":6024,"threads_started":1}
I20260926 21:23:00.661036 17037 leader_election.cc:302] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:00.661499 17180 raft_consensus.cc:2781] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:00.661541 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad10bf2665dc4776916065a37401e1d7" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451" is_pre_election: true
I20260926 21:23:00.661648 17180 raft_consensus.cc:482] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:00.661749 17180 raft_consensus.cc:3037] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:00.661829 16825 raft_consensus.cc:2454] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 0.
I20260926 21:23:00.665683 17180 raft_consensus.cc:504] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.666399 17180 leader_election.cc:288] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 election: Requested vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), bacd8b26434a406bacd598359525b51c (127.14.142.2:41869)
I20260926 21:23:00.666963 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad10bf2665dc4776916065a37401e1d7" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451"
I20260926 21:23:00.667001 16969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad10bf2665dc4776916065a37401e1d7" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bacd8b26434a406bacd598359525b51c"
I20260926 21:23:00.667201 16825 raft_consensus.cc:3037] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:00.667239 16969 raft_consensus.cc:3037] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:00.669970 16969 raft_consensus.cc:2454] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 1.
I20260926 21:23:00.670120 16825 raft_consensus.cc:2454] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 1.
I20260926 21:23:00.670531 17037 leader_election.cc:302] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:00.670949 17180 raft_consensus.cc:2781] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:00.671443 17180 raft_consensus.cc:686] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 1 LEADER]: Becoming Leader. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Running, Role: LEADER
I20260926 21:23:00.671900 17180 consensus_queue.cc:237] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:00.676091 16668 catalog_manager.cc:5495] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 reported cstate change: term changed from 0 to 1, leader changed from <none> to 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "026353374fa64cafbcbc7fafaee5c724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } health_report { overall_health: HEALTHY } } }
I20260926 21:23:00.722478 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.708604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.14.142.3:43903)
0926 21:23:00.715885 (+ 7281us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.715890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.715915 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:23:00.716009 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.716013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.716103 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.716342 (+ 239us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:00.716923 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.717902 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.719442 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.719915 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.720066 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.720867 (+ 801us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.721635 (+ 768us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.721818 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.721921 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.722011 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7159,"thread_start_us":6966,"threads_started":1}
I20260926 21:23:00.743798 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.734244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41869 (local address 127.14.142.3:48813)
0926 21:23:00.737208 (+ 2964us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:00.737228 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:00.737485 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:00.737913 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:00.737916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:00.737927 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:23:00.738241 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.738249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.739904 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.739909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.741232 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.741245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.742419 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.742424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.742625 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.742687 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.743212 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.743223 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 21:23:00.743309 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 21:23:00.743436 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2772,"thread_start_us":2576,"threads_started":1}
I20260926 21:23:00.743916 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.734331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48813 (local address 127.14.142.2:41869)
0926 21:23:00.734504 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.734509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.737398 (+ 2889us) server_negotiation.cc:407] Connection header received
0926 21:23:00.737627 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.737633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.737775 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.737899 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:00.738418 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.739714 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.741400 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.742203 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.742348 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.742879 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.743058 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.743170 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.743363 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.743482 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 21:23:00.753062 16969 raft_consensus.cc:1261] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Refusing update from remote peer 026353374fa64cafbcbc7fafaee5c724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:00.753247 16825 raft_consensus.cc:1261] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Refusing update from remote peer 026353374fa64cafbcbc7fafaee5c724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:00.753947 17227 consensus_queue.cc:1035] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:00.754330 17180 consensus_queue.cc:1035] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:00.760087 17201 mvcc.cc:205] Tried to move back new op lower bound from 7333715069957455872 to 7333715069637349376. Current Snapshot: MvccSnapshot[applied={T|T < 7333715069957455872}]
I20260926 21:23:00.791510 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.784632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43903 (local address 127.14.142.2:42893)
0926 21:23:00.784992 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:00.785012 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:00.785159 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:00.785622 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:00.785626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:00.785640 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:23:00.785936 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.785948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.787417 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.787422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.789257 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:00.789276 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.790295 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.790300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:00.790444 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.790491 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.790957 (+ 466us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.790968 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 21:23:00.791051 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 21:23:00.791174 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260926 21:23:00.791527 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:00.784746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:42893 (local address 127.14.142.3:43903)
0926 21:23:00.784930 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:00.784937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:00.785105 (+ 168us) server_negotiation.cc:407] Connection header received
0926 21:23:00.785312 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:00.785319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:00.785409 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:00.785525 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:00.786101 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.787216 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.789457 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:00.790107 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:00.790247 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:00.790667 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:00.790814 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:00.790911 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:00.791108 (+ 197us) server_negotiation.cc:299] Negotiation successful
0926 21:23:00.791195 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 21:23:00.793159 17201 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 21:23:02.427652 16668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:02.427910 16668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:02.433058 16668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Sending DeleteTablet for 3 replicas of tablet ad10bf2665dc4776916065a37401e1d7
I20260926 21:23:02.434031 16804 tablet_service.cc:1514] Processing DeleteTablet for tablet ad10bf2665dc4776916065a37401e1d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:02 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:54250
I20260926 21:23:02.434123 16944 tablet_service.cc:1514] Processing DeleteTablet for tablet ad10bf2665dc4776916065a37401e1d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:02 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55528
I20260926 21:23:02.434324 17084 tablet_service.cc:1514] Processing DeleteTablet for tablet ad10bf2665dc4776916065a37401e1d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:02 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:52844
I20260926 21:23:02.434813 17261 tablet_replica.cc:330] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: stopping tablet replica
I20260926 21:23:02.435009 17262 tablet_replica.cc:330] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: stopping tablet replica
I20260926 21:23:02.435037 17261 raft_consensus.cc:2229] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:02.435225 17262 raft_consensus.cc:2229] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:02.435323 17261 raft_consensus.cc:2258] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:02.435495 17262 raft_consensus.cc:2258] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:02.436025 17263 tablet_replica.cc:330] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: stopping tablet replica
I20260926 21:23:02.436244 17263 raft_consensus.cc:2229] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:02.436847 17263 raft_consensus.cc:2258] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:02.437183 17262 ts_tablet_manager.cc:1905] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:02.438400 17263 ts_tablet_manager.cc:1905] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:02.438442 17261 ts_tablet_manager.cc:1905] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:02.444226 17262 ts_tablet_manager.cc:1918] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 21:23:02.444423 17262 log.cc:1196] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/ad10bf2665dc4776916065a37401e1d7
I20260926 21:23:02.444977 17262 ts_tablet_manager.cc:1939] T ad10bf2665dc4776916065a37401e1d7 P bacd8b26434a406bacd598359525b51c: Deleting consensus metadata
I20260926 21:23:02.445231 17263 ts_tablet_manager.cc:1918] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 21:23:02.445598 17263 log.cc:1196] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/ad10bf2665dc4776916065a37401e1d7
I20260926 21:23:02.446271 17263 ts_tablet_manager.cc:1939] T ad10bf2665dc4776916065a37401e1d7 P 026353374fa64cafbcbc7fafaee5c724: Deleting consensus metadata
I20260926 21:23:02.446652 16655 catalog_manager.cc:4841] TS bacd8b26434a406bacd598359525b51c (127.14.142.2:41869): tablet ad10bf2665dc4776916065a37401e1d7 (table test-table [id=1b0b1a32e9f04e3191f36f857d33aaad]) successfully deleted
I20260926 21:23:02.447417 16652 catalog_manager.cc:4841] TS 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903): tablet ad10bf2665dc4776916065a37401e1d7 (table test-table [id=1b0b1a32e9f04e3191f36f857d33aaad]) successfully deleted
I20260926 21:23:02.447682 17261 ts_tablet_manager.cc:1918] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 21:23:02.447846 17261 log.cc:1196] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/ad10bf2665dc4776916065a37401e1d7
I20260926 21:23:02.448321 17261 ts_tablet_manager.cc:1939] T ad10bf2665dc4776916065a37401e1d7 P a3058222abd04ed58d85471e75fde451: Deleting consensus metadata
I20260926 21:23:02.449672 16654 catalog_manager.cc:4841] TS a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461): tablet ad10bf2665dc4776916065a37401e1d7 (table test-table [id=1b0b1a32e9f04e3191f36f857d33aaad]) successfully deleted
Sep 26 21:23:02 dist-test-slave-4d5t krb5kdc[16623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 21:23:02 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:02.496894 17271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:02.479466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.14.142.62:38315)
0926 21:23:02.480320 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:02.480326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:02.480392 (+ 66us) server_negotiation.cc:407] Connection header received
0926 21:23:02.480460 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:02.480464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:02.480537 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:02.480726 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:02.482505 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:02.483368 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:02.484935 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:02.485391 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:02.485493 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:02.490058 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:02.490065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:02.490069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:02.490128 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:02.493558 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:02.494601 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:02.494605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:02.494608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:02.494773 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:02.495315 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:02.495320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:02.495322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:02.495643 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:02.495855 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:02.496196 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 21:23:02.496384 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":405,"threads_started":1}
I20260926 21:23:02.501384 16668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:02.502209 16668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:02.509364 16944 tablet_service.cc:1467] Processing CreateTablet for tablet ad6161f8284249bb933fa230bdb62ccc (DEFAULT_TABLE table=test-table [id=7a8c22076507448a803afaa582c472b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:02.509308 16804 tablet_service.cc:1467] Processing CreateTablet for tablet ad6161f8284249bb933fa230bdb62ccc (DEFAULT_TABLE table=test-table [id=7a8c22076507448a803afaa582c472b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:02.509706 17084 tablet_service.cc:1467] Processing CreateTablet for tablet ad6161f8284249bb933fa230bdb62ccc (DEFAULT_TABLE table=test-table [id=7a8c22076507448a803afaa582c472b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:02.509933 16944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6161f8284249bb933fa230bdb62ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:02.509927 16804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6161f8284249bb933fa230bdb62ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:02.510203 17084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad6161f8284249bb933fa230bdb62ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:02.522734 17274 tablet_bootstrap.cc:492] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Bootstrap starting.
I20260926 21:23:02.523979 17272 tablet_bootstrap.cc:492] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Bootstrap starting.
I20260926 21:23:02.524448 17273 tablet_bootstrap.cc:492] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Bootstrap starting.
I20260926 21:23:02.526074 17274 tablet_bootstrap.cc:654] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:02.526273 17272 tablet_bootstrap.cc:654] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:02.526721 17273 tablet_bootstrap.cc:654] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:02.532204 17273 tablet_bootstrap.cc:492] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: No bootstrap required, opened a new log
I20260926 21:23:02.532352 17274 tablet_bootstrap.cc:492] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: No bootstrap required, opened a new log
I20260926 21:23:02.532372 17273 ts_tablet_manager.cc:1397] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260926 21:23:02.532510 17274 ts_tablet_manager.cc:1397] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260926 21:23:02.533360 17272 tablet_bootstrap.cc:492] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: No bootstrap required, opened a new log
I20260926 21:23:02.533293 17273 raft_consensus.cc:348] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.533282 17274 raft_consensus.cc:348] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.533517 17272 ts_tablet_manager.cc:1397] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260926 21:23:02.533529 17274 raft_consensus.cc:374] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:02.533532 17273 raft_consensus.cc:374] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:02.533619 17273 raft_consensus.cc:729] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3058222abd04ed58d85471e75fde451, State: Initialized, Role: FOLLOWER
I20260926 21:23:02.533622 17274 raft_consensus.cc:729] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacd8b26434a406bacd598359525b51c, State: Initialized, Role: FOLLOWER
I20260926 21:23:02.533877 17273 consensus_queue.cc:260] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.533869 17274 consensus_queue.cc:260] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.534318 17272 raft_consensus.cc:348] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.534684 17272 raft_consensus.cc:374] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:02.534803 17272 raft_consensus.cc:729] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Initialized, Role: FOLLOWER
I20260926 21:23:02.535753 17274 ts_tablet_manager.cc:1428] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 21:23:02.536478 17272 consensus_queue.cc:260] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.537309 17272 ts_tablet_manager.cc:1428] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260926 21:23:02.537955 17273 ts_tablet_manager.cc:1428] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260926 21:23:02.599303 17254 raft_consensus.cc:482] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:02.599510 17254 raft_consensus.cc:504] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.600327 17254 leader_election.cc:288] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), bacd8b26434a406bacd598359525b51c (127.14.142.2:41869)
I20260926 21:23:02.600452 17247 raft_consensus.cc:482] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:02.600750 17247 raft_consensus.cc:504] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.600948 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451" is_pre_election: true
I20260926 21:23:02.601006 16969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bacd8b26434a406bacd598359525b51c" is_pre_election: true
I20260926 21:23:02.601189 16824 raft_consensus.cc:2454] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 0.
I20260926 21:23:02.601250 16969 raft_consensus.cc:2454] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 0.
I20260926 21:23:02.601522 17247 leader_election.cc:288] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:23:02.601660 17036 leader_election.cc:302] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, a3058222abd04ed58d85471e75fde451; no voters:
I20260926 21:23:02.601950 17254 raft_consensus.cc:2781] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:02.602077 17254 raft_consensus.cc:482] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:02.602020 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451" is_pre_election: true
I20260926 21:23:02.602169 17254 raft_consensus.cc:3037] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:02.602108 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724" is_pre_election: true
I20260926 21:23:02.602221 16824 raft_consensus.cc:2454] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:23:02.602574 16897 leader_election.cc:302] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3058222abd04ed58d85471e75fde451, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:02.602823 17247 raft_consensus.cc:2781] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:02.602948 17247 raft_consensus.cc:482] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:02.603034 17247 raft_consensus.cc:3037] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:02.605353 17254 raft_consensus.cc:504] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.605865 17108 raft_consensus.cc:2379] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bacd8b26434a406bacd598359525b51c in current term 1: Already voted for candidate 026353374fa64cafbcbc7fafaee5c724 in this term.
I20260926 21:23:02.605949 17247 raft_consensus.cc:504] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.606292 17254 leader_election.cc:288] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 election: Requested vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), bacd8b26434a406bacd598359525b51c (127.14.142.2:41869)
I20260926 21:23:02.606617 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451"
I20260926 21:23:02.606814 16824 raft_consensus.cc:3037] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:02.606775 16969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "026353374fa64cafbcbc7fafaee5c724" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bacd8b26434a406bacd598359525b51c"
I20260926 21:23:02.606963 17247 leader_election.cc:288] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Requested vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:23:02.607051 16969 raft_consensus.cc:2379] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 026353374fa64cafbcbc7fafaee5c724 in current term 1: Already voted for candidate bacd8b26434a406bacd598359525b51c in this term.
I20260926 21:23:02.607698 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724"
I20260926 21:23:02.607759 16825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad6161f8284249bb933fa230bdb62ccc" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451"
I20260926 21:23:02.607987 17108 raft_consensus.cc:2379] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bacd8b26434a406bacd598359525b51c in current term 1: Already voted for candidate 026353374fa64cafbcbc7fafaee5c724 in this term.
I20260926 21:23:02.608419 16895 leader_election.cc:302] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bacd8b26434a406bacd598359525b51c; no voters: 026353374fa64cafbcbc7fafaee5c724, a3058222abd04ed58d85471e75fde451
I20260926 21:23:02.608877 17247 raft_consensus.cc:2726] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 21:23:02.609696 16824 raft_consensus.cc:2454] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 026353374fa64cafbcbc7fafaee5c724 in term 1.
I20260926 21:23:02.610285 17036 leader_election.cc:302] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, a3058222abd04ed58d85471e75fde451; no voters: bacd8b26434a406bacd598359525b51c
I20260926 21:23:02.610636 17254 raft_consensus.cc:2781] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:02.610795 17254 raft_consensus.cc:686] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 LEADER]: Becoming Leader. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Running, Role: LEADER
I20260926 21:23:02.611069 17254 consensus_queue.cc:237] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.614377 16668 catalog_manager.cc:5495] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 reported cstate change: term changed from 0 to 1, leader changed from <none> to 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "026353374fa64cafbcbc7fafaee5c724" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } health_report { overall_health: HEALTHY } } }
I20260926 21:23:02.643151 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:02.636317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.14.142.3:43903)
0926 21:23:02.637138 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:02.637143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:02.637166 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:23:02.637240 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:02.637244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:02.637318 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:02.637417 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:02.637950 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:02.638866 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:02.640596 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:02.641048 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:02.641165 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:02.641792 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:02.641922 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:02.642017 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:02.642298 (+ 281us) server_negotiation.cc:299] Negotiation successful
0926 21:23:02.642373 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":504,"threads_started":1}
I20260926 21:23:02.653692 16969 raft_consensus.cc:1261] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Refusing update from remote peer 026353374fa64cafbcbc7fafaee5c724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:02.653812 16824 raft_consensus.cc:1261] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Refusing update from remote peer 026353374fa64cafbcbc7fafaee5c724: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:02.654310 17235 consensus_queue.cc:1035] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:02.654651 17254 consensus_queue.cc:1035] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:02.710561 16665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:02.710855 16665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:02.714000 16665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Sending DeleteTablet for 3 replicas of tablet ad6161f8284249bb933fa230bdb62ccc
I20260926 21:23:02.714915 16944 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6161f8284249bb933fa230bdb62ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:02 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55528
I20260926 21:23:02.714937 16804 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6161f8284249bb933fa230bdb62ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:02 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:54250
I20260926 21:23:02.715016 17084 tablet_service.cc:1514] Processing DeleteTablet for tablet ad6161f8284249bb933fa230bdb62ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:02 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:52844
I20260926 21:23:02.715204 17262 tablet_replica.cc:330] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: stopping tablet replica
I20260926 21:23:02.715260 17261 tablet_replica.cc:330] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: stopping tablet replica
I20260926 21:23:02.715334 17263 tablet_replica.cc:330] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: stopping tablet replica
I20260926 21:23:02.715407 17262 raft_consensus.cc:2229] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:02.715472 17261 raft_consensus.cc:2229] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:02.715536 17263 raft_consensus.cc:2229] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:02.715616 17262 raft_consensus.cc:2258] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:02.715692 17261 raft_consensus.cc:2258] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:02.715873 17263 raft_consensus.cc:2258] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:02.716930 17261 ts_tablet_manager.cc:1905] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:02.717007 17263 ts_tablet_manager.cc:1905] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:02.717010 16665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 21:23:02.717423 17262 ts_tablet_manager.cc:1905] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 21:23:02.717693 16665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:02.723855 17261 ts_tablet_manager.cc:1918] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:02.724080 17261 log.cc:1196] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/ad6161f8284249bb933fa230bdb62ccc
I20260926 21:23:02.724519 17262 ts_tablet_manager.cc:1918] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:02.724740 17262 log.cc:1196] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/ad6161f8284249bb933fa230bdb62ccc
I20260926 21:23:02.724766 17261 ts_tablet_manager.cc:1939] T ad6161f8284249bb933fa230bdb62ccc P a3058222abd04ed58d85471e75fde451: Deleting consensus metadata
I20260926 21:23:02.725212 17262 ts_tablet_manager.cc:1939] T ad6161f8284249bb933fa230bdb62ccc P bacd8b26434a406bacd598359525b51c: Deleting consensus metadata
I20260926 21:23:02.725976 17263 ts_tablet_manager.cc:1918] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:02.726159 17263 log.cc:1196] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/ad6161f8284249bb933fa230bdb62ccc
I20260926 21:23:02.726361 16655 catalog_manager.cc:4841] TS bacd8b26434a406bacd598359525b51c (127.14.142.2:41869): tablet ad6161f8284249bb933fa230bdb62ccc (table test-table [id=7a8c22076507448a803afaa582c472b3]) successfully deleted
I20260926 21:23:02.726598 17263 ts_tablet_manager.cc:1939] T ad6161f8284249bb933fa230bdb62ccc P 026353374fa64cafbcbc7fafaee5c724: Deleting consensus metadata
I20260926 21:23:02.726958 17084 tablet_service.cc:1467] Processing CreateTablet for tablet 4c7c717bd7934c378ae8bbda52a3d566 (DEFAULT_TABLE table=test-table [id=dd4c92f8c8f44b54b664abdcd434085a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:02.726900 16944 tablet_service.cc:1467] Processing CreateTablet for tablet 4c7c717bd7934c378ae8bbda52a3d566 (DEFAULT_TABLE table=test-table [id=dd4c92f8c8f44b54b664abdcd434085a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:02.727417 17084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c7c717bd7934c378ae8bbda52a3d566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:02.727763 16944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c7c717bd7934c378ae8bbda52a3d566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:02.727959 16652 catalog_manager.cc:4841] TS 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903): tablet ad6161f8284249bb933fa230bdb62ccc (table test-table [id=7a8c22076507448a803afaa582c472b3]) successfully deleted
I20260926 21:23:02.728623 16654 catalog_manager.cc:4841] TS a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461): tablet ad6161f8284249bb933fa230bdb62ccc (table test-table [id=7a8c22076507448a803afaa582c472b3]) successfully deleted
I20260926 21:23:02.729338 16804 tablet_service.cc:1467] Processing CreateTablet for tablet 4c7c717bd7934c378ae8bbda52a3d566 (DEFAULT_TABLE table=test-table [id=dd4c92f8c8f44b54b664abdcd434085a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:02.730055 16804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c7c717bd7934c378ae8bbda52a3d566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:02.733186 17272 tablet_bootstrap.cc:492] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Bootstrap starting.
I20260926 21:23:02.735060 17272 tablet_bootstrap.cc:654] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:02.737274 17273 tablet_bootstrap.cc:492] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Bootstrap starting.
I20260926 21:23:02.737391 17272 tablet_bootstrap.cc:492] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: No bootstrap required, opened a new log
I20260926 21:23:02.737560 17272 ts_tablet_manager.cc:1397] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260926 21:23:02.738260 17272 raft_consensus.cc:348] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.738487 17272 raft_consensus.cc:374] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:02.738590 17272 raft_consensus.cc:729] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 026353374fa64cafbcbc7fafaee5c724, State: Initialized, Role: FOLLOWER
I20260926 21:23:02.738858 17272 consensus_queue.cc:260] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.739054 17274 tablet_bootstrap.cc:492] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Bootstrap starting.
I20260926 21:23:02.739074 17273 tablet_bootstrap.cc:654] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:02.739500 17272 ts_tablet_manager.cc:1428] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 21:23:02.740963 17274 tablet_bootstrap.cc:654] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:02.742708 17273 tablet_bootstrap.cc:492] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: No bootstrap required, opened a new log
I20260926 21:23:02.742941 17273 ts_tablet_manager.cc:1397] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260926 21:23:02.743980 17274 tablet_bootstrap.cc:492] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: No bootstrap required, opened a new log
I20260926 21:23:02.744168 17274 ts_tablet_manager.cc:1397] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 21:23:02.744469 17273 raft_consensus.cc:348] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.744809 17273 raft_consensus.cc:374] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:02.744907 17273 raft_consensus.cc:729] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3058222abd04ed58d85471e75fde451, State: Initialized, Role: FOLLOWER
I20260926 21:23:02.745141 17274 raft_consensus.cc:348] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.745182 17273 consensus_queue.cc:260] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.745421 17274 raft_consensus.cc:374] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:02.745517 17274 raft_consensus.cc:729] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bacd8b26434a406bacd598359525b51c, State: Initialized, Role: FOLLOWER
I20260926 21:23:02.745920 17273 ts_tablet_manager.cc:1428] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260926 21:23:02.745784 17274 consensus_queue.cc:260] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.746381 17274 ts_tablet_manager.cc:1428] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 21:23:02.792243 17247 raft_consensus.cc:482] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:02.792490 17247 raft_consensus.cc:504] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.793256 17247 leader_election.cc:288] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:23:02.793952 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7c717bd7934c378ae8bbda52a3d566" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451" is_pre_election: true
I20260926 21:23:02.794234 16824 raft_consensus.cc:2454] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:23:02.794130 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7c717bd7934c378ae8bbda52a3d566" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724" is_pre_election: true
I20260926 21:23:02.794467 17108 raft_consensus.cc:2454] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 0.
I20260926 21:23:02.794803 16897 leader_election.cc:302] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3058222abd04ed58d85471e75fde451, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:02.795181 17247 raft_consensus.cc:2781] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:02.795305 17247 raft_consensus.cc:482] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:02.795393 17247 raft_consensus.cc:3037] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:02.797847 17247 raft_consensus.cc:504] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.798672 17247 leader_election.cc:288] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Requested vote from peers a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461), 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903)
I20260926 21:23:02.799229 17108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7c717bd7934c378ae8bbda52a3d566" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "026353374fa64cafbcbc7fafaee5c724"
I20260926 21:23:02.799249 16824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7c717bd7934c378ae8bbda52a3d566" candidate_uuid: "bacd8b26434a406bacd598359525b51c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3058222abd04ed58d85471e75fde451"
I20260926 21:23:02.799486 16824 raft_consensus.cc:3037] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:02.799577 17108 raft_consensus.cc:3037] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:02.803195 16824 raft_consensus.cc:2454] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 1.
I20260926 21:23:02.803251 17108 raft_consensus.cc:2454] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bacd8b26434a406bacd598359525b51c in term 1.
I20260926 21:23:02.803833 16897 leader_election.cc:302] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 026353374fa64cafbcbc7fafaee5c724, a3058222abd04ed58d85471e75fde451, bacd8b26434a406bacd598359525b51c; no voters:
I20260926 21:23:02.804173 17247 raft_consensus.cc:2781] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:02.804394 17247 raft_consensus.cc:686] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 1 LEADER]: Becoming Leader. State: Replica: bacd8b26434a406bacd598359525b51c, State: Running, Role: LEADER
I20260926 21:23:02.804736 17247 consensus_queue.cc:237] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } }
I20260926 21:23:02.808290 16665 catalog_manager.cc:5495] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c reported cstate change: term changed from 0 to 1, leader changed from <none> to bacd8b26434a406bacd598359525b51c (127.14.142.2). New cstate: current_term: 1 leader_uuid: "bacd8b26434a406bacd598359525b51c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bacd8b26434a406bacd598359525b51c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:02.839929 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:02.834201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.14.142.2:41869)
0926 21:23:02.834747 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:02.834752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:02.834774 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:23:02.834827 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:02.834831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:02.834893 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:02.834983 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:02.835414 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:02.836180 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:02.837758 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:02.838381 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:02.838545 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:02.839010 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:02.839199 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:02.839296 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:02.839484 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 21:23:02.839561 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":529,"threads_started":1}
I20260926 21:23:02.849483 17108 raft_consensus.cc:1261] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Refusing update from remote peer bacd8b26434a406bacd598359525b51c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:02.849639 16824 raft_consensus.cc:1261] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Refusing update from remote peer bacd8b26434a406bacd598359525b51c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:02.850168 17242 consensus_queue.cc:1035] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [LEADER]: Connected to new peer: Peer: permanent_uuid: "026353374fa64cafbcbc7fafaee5c724" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:02.850569 17239 consensus_queue.cc:1035] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3058222abd04ed58d85471e75fde451" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 21:23:02.873387 17240 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 21:23:04.467451 16665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:04.467682 16665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:04.471863 16665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 362ded96812b46d8adb98e7335ea78d3: Sending DeleteTablet for 3 replicas of tablet 4c7c717bd7934c378ae8bbda52a3d566
I20260926 21:23:04.472800 16943 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c7c717bd7934c378ae8bbda52a3d566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:04 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55528
I20260926 21:23:04.472813 16804 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c7c717bd7934c378ae8bbda52a3d566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:04 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:54250
I20260926 21:23:04.472888 17084 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c7c717bd7934c378ae8bbda52a3d566 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:04 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:52844
I20260926 21:23:04.473842 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16739
I20260926 21:23:04.485214 17317 tablet_replica.cc:330] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: stopping tablet replica
I20260926 21:23:04.485345 17316 tablet_replica.cc:330] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: stopping tablet replica
I20260926 21:23:04.485462 17315 tablet_replica.cc:330] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: stopping tablet replica
I20260926 21:23:04.485488 17317 raft_consensus.cc:2229] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:04.485632 17316 raft_consensus.cc:2229] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:04.485709 17315 raft_consensus.cc:2229] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:04.485852 17317 raft_consensus.cc:2258] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:04.486006 17315 raft_consensus.cc:2258] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:04.486261 17316 raft_consensus.cc:2258] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:04.487587 17317 ts_tablet_manager.cc:1905] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:04.487634 17316 ts_tablet_manager.cc:1905] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:04.488199 17315 ts_tablet_manager.cc:1905] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:04.496203 17317 ts_tablet_manager.cc:1918] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 21:23:04.496413 17317 log.cc:1196] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/4c7c717bd7934c378ae8bbda52a3d566
I20260926 21:23:04.496642 17316 ts_tablet_manager.cc:1918] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 21:23:04.496797 17316 log.cc:1196] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/4c7c717bd7934c378ae8bbda52a3d566
I20260926 21:23:04.497016 17317 ts_tablet_manager.cc:1939] T 4c7c717bd7934c378ae8bbda52a3d566 P 026353374fa64cafbcbc7fafaee5c724: Deleting consensus metadata
I20260926 21:23:04.497275 17316 ts_tablet_manager.cc:1939] T 4c7c717bd7934c378ae8bbda52a3d566 P bacd8b26434a406bacd598359525b51c: Deleting consensus metadata
I20260926 21:23:04.497816 17315 ts_tablet_manager.cc:1918] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 21:23:04.497982 17315 log.cc:1196] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/4c7c717bd7934c378ae8bbda52a3d566
I20260926 21:23:04.498440 17315 ts_tablet_manager.cc:1939] T 4c7c717bd7934c378ae8bbda52a3d566 P a3058222abd04ed58d85471e75fde451: Deleting consensus metadata
I20260926 21:23:04.498456 16655 catalog_manager.cc:4841] TS bacd8b26434a406bacd598359525b51c (127.14.142.2:41869): tablet 4c7c717bd7934c378ae8bbda52a3d566 (table test-table [id=dd4c92f8c8f44b54b664abdcd434085a]) successfully deleted
I20260926 21:23:04.498401 16652 catalog_manager.cc:4841] TS 026353374fa64cafbcbc7fafaee5c724 (127.14.142.3:43903): tablet 4c7c717bd7934c378ae8bbda52a3d566 (table test-table [id=dd4c92f8c8f44b54b664abdcd434085a]) successfully deleted
I20260926 21:23:04.499450 16654 catalog_manager.cc:4841] TS a3058222abd04ed58d85471e75fde451 (127.14.142.1:36461): tablet 4c7c717bd7934c378ae8bbda52a3d566 (table test-table [id=dd4c92f8c8f44b54b664abdcd434085a]) successfully deleted
Sep 26 21:23:04 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:04.511863 17319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:04.476182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.14.142.1:36461)
0926 21:23:04.487358 (+ 11176us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:04.487364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:04.487389 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:23:04.487452 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:04.487457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:04.487530 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:04.487663 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:04.489572 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:04.491246 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:04.493818 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:04.494664 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:04.494824 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:04.505388 (+ 10564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:04.505404 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:04.505421 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:04.505463 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:04.508887 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:04.509693 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:04.509701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:04.509703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:04.509839 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:04.510253 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:04.510257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:04.510259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:04.510626 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:04.510965 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:04.511270 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 21:23:04.511504 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11026,"thread_start_us":11122,"threads_started":1}
W20260926 21:23:04.513242 16870 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54254
W20260926 21:23:04.514191 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16739: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:23:04.514662 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16739
I20260926 21:23:04.533792 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16879
Sep 26 21:23:04 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:23:04.552973 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:04.534742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.14.142.2:41869)
0926 21:23:04.535476 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:04.535482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:04.535509 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:04.535590 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:04.535595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:04.535692 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:04.535838 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:04.537734 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:04.538624 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:04.540496 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:04.541122 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:04.541251 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:04.545852 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:04.545874 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:04.545897 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:04.545963 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:04.549543 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:04.550375 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:04.550383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:04.550385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:04.550556 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:04.551104 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:04.551109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:04.551111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:04.551541 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:04.551790 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:04.552187 (+ 397us) server_negotiation.cc:299] Negotiation successful
0926 21:23:04.552451 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":357,"threads_started":1}
W20260926 21:23:04.553963 17009 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55582
W20260926 21:23:04.554773 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16879: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:23:04.554962 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16879
I20260926 21:23:04.581043 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17018
Sep 26 21:23:04 dist-test-slave-4d5t krb5kdc[16623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457782, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:23:04.598697 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:04.582048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.14.142.3:43903)
0926 21:23:04.582625 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:04.582630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:04.582648 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:23:04.582707 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:04.582710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:04.582782 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:04.582941 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:04.584254 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:04.585087 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:04.586380 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:04.586813 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:04.586965 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:04.592226 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:04.592244 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:04.592261 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:04.592318 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:04.595572 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:04.596419 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:04.596427 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:04.596429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:04.596639 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:04.597038 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:04.597045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:04.597048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:04.597416 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:04.597719 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:04.598043 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 21:23:04.598283 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":284,"threads_started":1}
W20260926 21:23:04.599449 17149 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52872
W20260926 21:23:04.600066 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17018: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:23:04.600276 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17018
I20260926 21:23:04.622761 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 16636
I20260926 21:23:04.624315 16708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:04.750579 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 16636
2026-09-26T21:23:04Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8905 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:04 dist-test-slave-4d5t krb5kdc[17325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:04 dist-test-slave-4d5t krb5kdc[17325](info): set up 2 sockets
Sep 26 21:23:04 dist-test-slave-4d5t krb5kdc[17325](info): commencing operation
krb5kdc: starting...
W20260926 21:23:05.885735 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:06 dist-test-slave-4d5t krb5kdc[17325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:06.095746 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:45479
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:45479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:06.442077 17338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:06.442399 17338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:06.442553 17338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:06.450080 17338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:06.450209 17338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:06.450302 17338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:06.450388 17338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:06.450455 17338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:06.461705 17338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:45479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:45479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:06.463140 17338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:06.464541 17338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:06.478078 17344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:06.478199 17346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:06.478782 17343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:06.479203 17338 server_base.cc:1034] running on GCE node
I20260926 21:23:06.480209 17338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:06.482025 17338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:06.483309 17338 hybrid_clock.cc:648] HybridClock initialized: now 1790457786483237 us; error 59 us; skew 500 ppm
I20260926 21:23:06.483716 17338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:06 dist-test-slave-4d5t krb5kdc[17325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457786, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:06.486999 17338 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:06.488636 17338 webserver.cc:460] Webserver started at http://127.14.142.62:39489/ using document root <none> and password file <none>
I20260926 21:23:06.489326 17338 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:06.489437 17338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:06.489694 17338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:06.492848 17338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "6b0ce5009453493e84b65e3626f7a104"
format_stamp: "Formatted at 2026-09-26 21:23:06 on dist-test-slave-4d5t"
server_key: "b0899d6e49cb488cf112f74d4acc5629"
server_key_iv: "92dc3217fe4b39fd440673b12ddc37a1"
server_key_version: "encryptionkey@0"
I20260926 21:23:06.493625 17338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "6b0ce5009453493e84b65e3626f7a104"
format_stamp: "Formatted at 2026-09-26 21:23:06 on dist-test-slave-4d5t"
server_key: "b0899d6e49cb488cf112f74d4acc5629"
server_key_iv: "92dc3217fe4b39fd440673b12ddc37a1"
server_key_version: "encryptionkey@0"
I20260926 21:23:06.500213 17338 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260926 21:23:06.505060 17353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:06.507133 17338 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:23:06.507344 17338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "6b0ce5009453493e84b65e3626f7a104"
format_stamp: "Formatted at 2026-09-26 21:23:06 on dist-test-slave-4d5t"
server_key: "b0899d6e49cb488cf112f74d4acc5629"
server_key_iv: "92dc3217fe4b39fd440673b12ddc37a1"
server_key_version: "encryptionkey@0"
I20260926 21:23:06.507539 17338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:06.522742 17338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:06.527824 17338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:06.528141 17338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:06.551849 17338 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:45479
I20260926 21:23:06.551894 17404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:45479 every 8 connection(s)
I20260926 21:23:06.553609 17338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:06.556928 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17338
I20260926 21:23:06.557181 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:06.557905 14904 external_mini_cluster.cc:1428] Setting key 9aa3b74463e162a6db38dd6760e67c03
I20260926 21:23:06.558115 17405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:06.575441 17405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104: Bootstrap starting.
Sep 26 21:23:06 dist-test-slave-4d5t krb5kdc[17325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:06.579461 17405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:06.581197 17405 log.cc:824] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:06.585654 17405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104: No bootstrap required, opened a new log
I20260926 21:23:06.586699 17408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:06.560083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50938 (local address 127.14.142.62:45479)
0926 21:23:06.560994 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:06.561006 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:06.561048 (+ 42us) server_negotiation.cc:407] Connection header received
0926 21:23:06.563327 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:06.563352 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:06.563844 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:06.564220 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:06.566337 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:06.568373 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:06.570758 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:06.571688 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:06.571864 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:06.577025 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:06.577035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:06.577054 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:06.577102 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:06.581161 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:06.582053 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:06.582062 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:06.582075 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:06.582289 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:06.582792 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:06.582799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:06.582808 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:06.583733 (+ 925us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:06.584080 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:06.584851 (+ 771us) server_negotiation.cc:299] Negotiation successful
0926 21:23:06.585346 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":444,"threads_started":1}
I20260926 21:23:06.594937 17405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ce5009453493e84b65e3626f7a104" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45479 } }
I20260926 21:23:06.595397 17405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:06.595504 17405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0ce5009453493e84b65e3626f7a104, State: Initialized, Role: FOLLOWER
I20260926 21:23:06.596127 17405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ce5009453493e84b65e3626f7a104" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45479 } }
I20260926 21:23:06.596369 17405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:06.596520 17405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:06.596755 17405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:06.693572 17405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ce5009453493e84b65e3626f7a104" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45479 } }
I20260926 21:23:06.694279 17405 leader_election.cc:302] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b0ce5009453493e84b65e3626f7a104; no voters:
I20260926 21:23:06.694985 17405 leader_election.cc:288] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:06.695209 17410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:06.696297 17410 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0ce5009453493e84b65e3626f7a104, State: Running, Role: LEADER
I20260926 21:23:06.697000 17410 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ce5009453493e84b65e3626f7a104" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45479 } }
I20260926 21:23:06.697423 17405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:06.701534 17411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b0ce5009453493e84b65e3626f7a104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ce5009453493e84b65e3626f7a104" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45479 } } }
I20260926 21:23:06.701154 17412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0ce5009453493e84b65e3626f7a104. Latest consensus state: current_term: 1 leader_uuid: "6b0ce5009453493e84b65e3626f7a104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0ce5009453493e84b65e3626f7a104" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45479 } } }
I20260926 21:23:06.702405 17411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:06.702893 17412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:06.706142 17418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:06.713040 17418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:06.722903 17418 catalog_manager.cc:1347] Generated new cluster ID: 12c70ff847594943a6653ba88accfc13
I20260926 21:23:06.723044 17418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:06.734910 17418 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:06.736860 17418 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:06.752339 17418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b0ce5009453493e84b65e3626f7a104: Generated new TSK 0
I20260926 21:23:06.753389 17418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
W20260926 21:23:08.067289 14904 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.14.142.1: real 1.309s user 0.000s sys 0.011s
I20260926 21:23:08.068643 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:45479
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:08.433441 17431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:08.433811 17431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:08.434005 17431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:08.441506 17431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:08.441671 17431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:08.441867 17431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:08.451719 17431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:45479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:08.453163 17431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:08.454556 17431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:08.468351 17437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:08.468528 17436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:08.468528 17439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:08.470109 17431 server_base.cc:1034] running on GCE node
I20260926 21:23:08.470717 17431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:08.471705 17431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:08.472919 17431 hybrid_clock.cc:648] HybridClock initialized: now 1790457788472876 us; error 26 us; skew 500 ppm
I20260926 21:23:08.473245 17431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:08 dist-test-slave-4d5t krb5kdc[17325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457788, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:08.476879 17431 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:23:08.478564 17431 webserver.cc:460] Webserver started at http://127.14.142.1:37095/ using document root <none> and password file <none>
I20260926 21:23:08.479288 17431 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:08.479398 17431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:08.479655 17431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:08.482738 17431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "01d1d9501c44442781a4ea62b58562af"
format_stamp: "Formatted at 2026-09-26 21:23:08 on dist-test-slave-4d5t"
server_key: "c0df00508c1ab0622c571783b5adc8d0"
server_key_iv: "4bea65bee3762f163782782053e54dd0"
server_key_version: "encryptionkey@0"
I20260926 21:23:08.483534 17431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "01d1d9501c44442781a4ea62b58562af"
format_stamp: "Formatted at 2026-09-26 21:23:08 on dist-test-slave-4d5t"
server_key: "c0df00508c1ab0622c571783b5adc8d0"
server_key_iv: "4bea65bee3762f163782782053e54dd0"
server_key_version: "encryptionkey@0"
I20260926 21:23:08.489084 17431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:08.492880 17446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:08.494989 17431 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:08.495285 17431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "01d1d9501c44442781a4ea62b58562af"
format_stamp: "Formatted at 2026-09-26 21:23:08 on dist-test-slave-4d5t"
server_key: "c0df00508c1ab0622c571783b5adc8d0"
server_key_iv: "4bea65bee3762f163782782053e54dd0"
server_key_version: "encryptionkey@0"
I20260926 21:23:08.495615 17431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:08.514006 17431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:08.519054 17431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:08.519340 17431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:08.520339 17431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:08.521828 17431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:08.521906 17431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:08.521997 17431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:08.522049 17431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:08.558259 17431 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:38629
I20260926 21:23:08.558297 17558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:38629 every 8 connection(s)
I20260926 21:23:08.559643 17431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:08.569994 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17431
I20260926 21:23:08.570248 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:08.570894 14904 external_mini_cluster.cc:1428] Setting key eaf52a7aa6309a48067d3da99f87e2fa
Sep 26 21:23:08 dist-test-slave-4d5t krb5kdc[17325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457788, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:08.585114 17561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:08.562659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:35611 (local address 127.14.142.62:45479)
0926 21:23:08.563296 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:08.563301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:08.564209 (+ 908us) server_negotiation.cc:407] Connection header received
0926 21:23:08.566318 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:08.566323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:08.566395 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:08.566508 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:08.568260 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:08.569325 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:08.571591 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:08.572170 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:08.572284 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:08.576696 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:08.576706 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:08.576713 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:08.576775 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:08.580835 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:08.582188 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:08.582197 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:08.582203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:08.582390 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:08.582859 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:08.582866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:08.582872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:08.583274 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:08.583452 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:08.584279 (+ 827us) server_negotiation.cc:299] Negotiation successful
0926 21:23:08.584591 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":401,"threads_started":1}
I20260926 21:23:08.586032 17562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:08.563072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:45479 (local address 127.14.142.1:35611)
0926 21:23:08.563995 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:08.564042 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:08.566022 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:08.566734 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:08.566743 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:08.567328 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:08.567949 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:08.567969 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:08.569583 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:08.569588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:08.571299 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:08.571330 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:08.572459 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:08.572468 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:08.572812 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:08.573721 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:08.573745 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:08.576336 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:08.581124 (+ 4788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:08.581136 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:08.581158 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:08.582022 (+ 864us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:08.582516 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:08.582520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:08.582523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:08.582738 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:08.583464 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:08.583471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:08.583861 (+ 390us) client_negotiation.cc:769] Sending connection context
0926 21:23:08.584155 (+ 294us) client_negotiation.cc:240] Negotiation successful
0926 21:23:08.584712 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":403,"threads_started":1}
I20260926 21:23:08.587951 17559 heartbeater.cc:347] Connected to a master server at 127.14.142.62:45479
I20260926 21:23:08.588299 17559 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:08.589214 17559 heartbeater.cc:511] Master 127.14.142.62:45479 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:23:08.592252 17370 ts_manager.cc:194] Registered new tserver with Master: 01d1d9501c44442781a4ea62b58562af (127.14.142.1:38629)
I20260926 21:23:08.593777 17370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:35611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:23:08.620457 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:45479
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:08.965456 17565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:08.965768 17565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:08.965932 17565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:08.973292 17565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:08.973421 17565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:08.973619 17565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:08.983625 17565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:45479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:08.985193 17565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:08.986605 17565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:09.000326 17570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:09.000794 17571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:09.001134 17573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:09.002770 17565 server_base.cc:1034] running on GCE node
I20260926 21:23:09.003851 17565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:09.005576 17565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:09.006841 17565 hybrid_clock.cc:648] HybridClock initialized: now 1790457789006797 us; error 43 us; skew 500 ppm
I20260926 21:23:09.007234 17565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457789, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:09.010818 17565 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:23:09.012696 17565 webserver.cc:460] Webserver started at http://127.14.142.2:45275/ using document root <none> and password file <none>
I20260926 21:23:09.013481 17565 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:09.013589 17565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:09.013861 17565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:09.016975 17565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "9613611cd6e1410faeb90abdaad00c2d"
format_stamp: "Formatted at 2026-09-26 21:23:09 on dist-test-slave-4d5t"
server_key: "8125e91f18311de997aa5b8d53ab0859"
server_key_iv: "b0aed56a3b099a625b6b2a39d988366f"
server_key_version: "encryptionkey@0"
I20260926 21:23:09.017757 17565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "9613611cd6e1410faeb90abdaad00c2d"
format_stamp: "Formatted at 2026-09-26 21:23:09 on dist-test-slave-4d5t"
server_key: "8125e91f18311de997aa5b8d53ab0859"
server_key_iv: "b0aed56a3b099a625b6b2a39d988366f"
server_key_version: "encryptionkey@0"
I20260926 21:23:09.023931 17565 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 21:23:09.028290 17580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:09.030054 17565 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 21:23:09.030270 17565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "9613611cd6e1410faeb90abdaad00c2d"
format_stamp: "Formatted at 2026-09-26 21:23:09 on dist-test-slave-4d5t"
server_key: "8125e91f18311de997aa5b8d53ab0859"
server_key_iv: "b0aed56a3b099a625b6b2a39d988366f"
server_key_version: "encryptionkey@0"
I20260926 21:23:09.030535 17565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:09.069520 17565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:09.074322 17565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:09.074648 17565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:09.075747 17565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:09.077248 17565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:09.077358 17565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:09.077519 17565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:09.077593 17565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:09.114480 17565 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:44521
I20260926 21:23:09.114526 17692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:44521 every 8 connection(s)
I20260926 21:23:09.115797 17565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:09.123538 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17565
I20260926 21:23:09.123744 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:09.124246 14904 external_mini_cluster.cc:1428] Setting key ab0fc335321b37c3bd8071a779812273
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457789, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:09.138466 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:09.118508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:59483 (local address 127.14.142.62:45479)
0926 21:23:09.119009 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:09.119015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:09.119934 (+ 919us) server_negotiation.cc:407] Connection header received
0926 21:23:09.121957 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:09.121962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:09.122043 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:09.122143 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:09.123793 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.124843 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.126886 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.127353 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.127513 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:09.131449 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:09.131454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:09.131458 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:09.131491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:09.134429 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:09.135328 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:09.135334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:09.135338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:09.135694 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:09.136343 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:09.136351 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:09.136355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:09.136751 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:09.136930 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:09.137786 (+ 856us) server_negotiation.cc:299] Negotiation successful
0926 21:23:09.138069 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":242,"threads_started":1}
I20260926 21:23:09.139869 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:09.118994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:45479 (local address 127.14.142.2:59483)
0926 21:23:09.119759 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:09.119799 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:09.121682 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:09.122362 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:09.122370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:09.122939 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:09.123556 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:09.123575 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.125033 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.125038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:09.126722 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:09.126735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.127622 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.127631 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:09.127876 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:09.128721 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:09.128753 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:09.131169 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:09.134593 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:09.134603 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:09.134624 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:09.135188 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:09.135875 (+ 687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:09.135881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:09.135885 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:09.136216 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:09.136890 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:09.136899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:09.137368 (+ 469us) client_negotiation.cc:769] Sending connection context
0926 21:23:09.137681 (+ 313us) client_negotiation.cc:240] Negotiation successful
0926 21:23:09.138279 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":300,"threads_started":1}
I20260926 21:23:09.141577 17693 heartbeater.cc:347] Connected to a master server at 127.14.142.62:45479
I20260926 21:23:09.141991 17693 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:09.142809 17693 heartbeater.cc:511] Master 127.14.142.62:45479 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:23:09.144505 17370 ts_manager.cc:194] Registered new tserver with Master: 9613611cd6e1410faeb90abdaad00c2d (127.14.142.2:44521)
I20260926 21:23:09.145434 17370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:59483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:23:09.173377 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:45479
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:09.513317 17699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:09.513691 17699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:09.513870 17699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:09.521261 17699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:09.521454 17699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:09.521651 17699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:09.532142 17699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:45479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:09.533607 17699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:09.534964 17699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:09.549955 17705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:09.549995 17707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:09.550916 17699 server_base.cc:1034] running on GCE node
W20260926 21:23:09.552685 17704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:09.553521 17699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:09.554654 17699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:09.555930 17699 hybrid_clock.cc:648] HybridClock initialized: now 1790457789555869 us; error 49 us; skew 500 ppm
I20260926 21:23:09.556322 17699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457789, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:09.564503 17699 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:23:09.566421 17699 webserver.cc:460] Webserver started at http://127.14.142.3:43375/ using document root <none> and password file <none>
I20260926 21:23:09.567175 17699 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:09.567332 17699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:09.567685 17699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:09.570858 17699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "bb1fda0dbc5446c6bd03e9fdee2677c8"
format_stamp: "Formatted at 2026-09-26 21:23:09 on dist-test-slave-4d5t"
server_key: "69b351deeeffd96c044f13b535c9b95b"
server_key_iv: "7fe8e1bb463b5c6abf861f2506be3155"
server_key_version: "encryptionkey@0"
I20260926 21:23:09.571692 17699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "bb1fda0dbc5446c6bd03e9fdee2677c8"
format_stamp: "Formatted at 2026-09-26 21:23:09 on dist-test-slave-4d5t"
server_key: "69b351deeeffd96c044f13b535c9b95b"
server_key_iv: "7fe8e1bb463b5c6abf861f2506be3155"
server_key_version: "encryptionkey@0"
I20260926 21:23:09.577544 17699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 21:23:09.581324 17714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:09.582954 17699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:09.583179 17699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "bb1fda0dbc5446c6bd03e9fdee2677c8"
format_stamp: "Formatted at 2026-09-26 21:23:09 on dist-test-slave-4d5t"
server_key: "69b351deeeffd96c044f13b535c9b95b"
server_key_iv: "7fe8e1bb463b5c6abf861f2506be3155"
server_key_version: "encryptionkey@0"
I20260926 21:23:09.583421 17699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:09.598063 17559 heartbeater.cc:503] Master 127.14.142.62:45479 was elected leader, sending a full tablet report...
I20260926 21:23:09.604252 17699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:09.609324 17699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:09.609643 17699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:09.610677 17699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:09.612318 17699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:09.612432 17699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:09.612618 17699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:09.612695 17699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260926 21:23:09.649144 17699 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:33253
I20260926 21:23:09.649190 17826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:33253 every 8 connection(s)
I20260926 21:23:09.650426 17699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:09.657038 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17699
I20260926 21:23:09.657224 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:09.657688 14904 external_mini_cluster.cc:1428] Setting key 43997bf4c4d5f3462e65399f1fe39371
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457789, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:09.674835 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:09.653110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:53423 (local address 127.14.142.62:45479)
0926 21:23:09.653580 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:09.653585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:09.654491 (+ 906us) server_negotiation.cc:407] Connection header received
0926 21:23:09.656597 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:09.656604 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:09.656711 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:09.656850 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:09.658935 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.660011 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.663298 (+ 3287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.663820 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.663971 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:09.668451 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:09.668456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:09.668459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:09.668500 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:09.671254 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:09.672183 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:09.672192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:09.672196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:09.672330 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:09.672881 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:09.672888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:09.672894 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:09.673194 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:09.673371 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:09.674150 (+ 779us) server_negotiation.cc:299] Negotiation successful
0926 21:23:09.674408 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":245,"threads_started":1}
I20260926 21:23:09.675946 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:09.653506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:45479 (local address 127.14.142.3:53423)
0926 21:23:09.654291 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:09.654344 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:09.656233 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:09.657156 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:09.657170 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:09.657913 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:09.658735 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:09.658767 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.660373 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.660377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:09.663069 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:09.663095 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:09.664128 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:09.664135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:09.664408 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:09.665289 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:09.665320 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:09.668094 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:09.671454 (+ 3360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:09.671464 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:09.671489 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:09.672032 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:09.672454 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:09.672459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:09.672463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:09.672762 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:09.673336 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:09.673346 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:09.673746 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 21:23:09.674049 (+ 303us) client_negotiation.cc:240] Negotiation successful
0926 21:23:09.674631 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":322,"threads_started":1}
I20260926 21:23:09.677712 17827 heartbeater.cc:347] Connected to a master server at 127.14.142.62:45479
I20260926 21:23:09.678120 17827 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:09.679183 17827 heartbeater.cc:511] Master 127.14.142.62:45479 requested a full tablet report, sending...
I20260926 21:23:09.681053 17370 ts_manager.cc:194] Registered new tserver with Master: bb1fda0dbc5446c6bd03e9fdee2677c8 (127.14.142.3:33253)
I20260926 21:23:09.682065 17370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:53423
I20260926 21:23:09.688653 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 21:23:09.722579 17829 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:09.719764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.14.142.62:45479)
0926 21:23:09.719998 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:09.720008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:09.720445 (+ 437us) server_negotiation.cc:407] Connection header received
0926 21:23:09.720711 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:09.720716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:09.720785 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:09.720909 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:09.722294 (+ 1385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50948: BlockingRecv error: recv got EOF from 127.0.0.1:50948 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:23:09.726953 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17431
W20260926 21:23:09.730530 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17431: Not authorized: Client connection negotiation failed: client connection to 127.14.142.1:38629: server requires authentication, but client does not have Kerberos credentials available
I20260926 21:23:09.730760 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17431
I20260926 21:23:09.746764 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17565
W20260926 21:23:09.749727 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17565: Not authorized: Client connection negotiation failed: client connection to 127.14.142.2:44521: server requires authentication, but client does not have Kerberos credentials available
I20260926 21:23:09.749938 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17565
I20260926 21:23:09.766003 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17699
W20260926 21:23:09.769124 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17699: Not authorized: Client connection negotiation failed: client connection to 127.14.142.3:33253: server requires authentication, but client does not have Kerberos credentials available
I20260926 21:23:09.769333 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17699
I20260926 21:23:09.785611 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17338
I20260926 21:23:09.786938 17400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:09.895010 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17338
2026-09-26T21:23:09Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5139 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17845](info): set up 2 sockets
Sep 26 21:23:09 dist-test-slave-4d5t krb5kdc[17845](info): commencing operation
krb5kdc: starting...
W20260926 21:23:11.001654 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:11 dist-test-slave-4d5t krb5kdc[17845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:11.221580 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46503
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:33339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:46503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:11.569375 17858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:11.569795 17858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:11.569973 17858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:11.578452 17858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:11.578622 17858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:11.578734 17858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:11.578840 17858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:11.578922 17858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:11.590668 17858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:33339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:46503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:11.592685 17858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:11.594143 17858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:11.607309 17863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:11.607352 17866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:11.607542 17864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:11.608665 17858 server_base.cc:1034] running on GCE node
I20260926 21:23:11.609498 17858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:11.611315 17858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:11.612742 17858 hybrid_clock.cc:648] HybridClock initialized: now 1790457791612535 us; error 52 us; skew 500 ppm
I20260926 21:23:11.613181 17858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:11 dist-test-slave-4d5t krb5kdc[17845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457791, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:11.616710 17858 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:11.618440 17858 webserver.cc:460] Webserver started at http://127.14.142.62:45987/ using document root <none> and password file <none>
I20260926 21:23:11.619179 17858 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:11.619335 17858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:11.619660 17858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:11.622802 17858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4"
format_stamp: "Formatted at 2026-09-26 21:23:11 on dist-test-slave-4d5t"
server_key: "9c1378b7cdedcfa9e29b6b7d0bfa2faf"
server_key_iv: "269711403683e8187738d0c791ae886c"
server_key_version: "encryptionkey@0"
I20260926 21:23:11.623638 17858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4"
format_stamp: "Formatted at 2026-09-26 21:23:11 on dist-test-slave-4d5t"
server_key: "9c1378b7cdedcfa9e29b6b7d0bfa2faf"
server_key_iv: "269711403683e8187738d0c791ae886c"
server_key_version: "encryptionkey@0"
I20260926 21:23:11.629244 17858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:11.633270 17873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:11.634893 17858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:23:11.635106 17858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4"
format_stamp: "Formatted at 2026-09-26 21:23:11 on dist-test-slave-4d5t"
server_key: "9c1378b7cdedcfa9e29b6b7d0bfa2faf"
server_key_iv: "269711403683e8187738d0c791ae886c"
server_key_version: "encryptionkey@0"
I20260926 21:23:11.635309 17858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:11.660142 17858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:11.665654 17858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:11.666064 17858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:11.689651 17858 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:46503
I20260926 21:23:11.689656 17924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:46503 every 8 connection(s)
I20260926 21:23:11.691603 17858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:11.695420 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17858
I20260926 21:23:11.695696 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:11.696523 14904 external_mini_cluster.cc:1428] Setting key b639529de7c7e583c8b1415721d00585
I20260926 21:23:11.697104 17925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 21:23:11 dist-test-slave-4d5t krb5kdc[17845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:11.715271 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4: Bootstrap starting.
I20260926 21:23:11.719643 17925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:11.720965 17925 log.cc:824] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:11.724270 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4: No bootstrap required, opened a new log
I20260926 21:23:11.726326 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:11.698848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.14.142.62:46503)
0926 21:23:11.699974 (+ 1126us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:11.699992 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:11.700045 (+ 53us) server_negotiation.cc:407] Connection header received
0926 21:23:11.702316 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:11.702344 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:11.702867 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:11.703300 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:11.705697 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:11.707376 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:11.709521 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:11.710468 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:11.710679 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:11.716422 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:11.716431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:11.716447 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:11.716499 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:11.720939 (+ 4440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:11.721995 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:11.722003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:11.722012 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:11.722276 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:11.722725 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:11.722729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:11.722732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:11.723481 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:11.723859 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:11.724527 (+ 668us) server_negotiation.cc:299] Negotiation successful
0926 21:23:11.725227 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851,"thread_start_us":540,"threads_started":1}
I20260926 21:23:11.730933 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:23:11.734149 17925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46503 } }
I20260926 21:23:11.734589 17925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:11.734709 17925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1efa5eef7b43c58d13b2c0c1f84db4, State: Initialized, Role: FOLLOWER
I20260926 21:23:11.735338 17925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46503 } }
I20260926 21:23:11.735615 17925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:11.735785 17925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:11.735984 17925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:11.738932 17925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46503 } }
I20260926 21:23:11.739558 17925 leader_election.cc:302] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e1efa5eef7b43c58d13b2c0c1f84db4; no voters:
I20260926 21:23:11.740278 17925 leader_election.cc:288] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:11.740425 17931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:11.743348 17925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:11.744930 17931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [term 1 LEADER]: Becoming Leader. State: Replica: 6e1efa5eef7b43c58d13b2c0c1f84db4, State: Running, Role: LEADER
I20260926 21:23:11.745891 17931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46503 } }
I20260926 21:23:11.750250 17932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46503 } } }
I20260926 21:23:11.750622 17932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:11.751267 17933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e1efa5eef7b43c58d13b2c0c1f84db4. Latest consensus state: current_term: 1 leader_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1efa5eef7b43c58d13b2c0c1f84db4" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46503 } } }
I20260926 21:23:11.751668 17933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:11.762446 17940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:11.768682 17940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:11.779130 17940 catalog_manager.cc:1347] Generated new cluster ID: 2217a736a7c745f392944a844f35c186
I20260926 21:23:11.779286 17940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:11.793934 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:11.778385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.14.142.62:46503)
0926 21:23:11.778566 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:11.778574 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:11.785150 (+ 6576us) server_negotiation.cc:407] Connection header received
0926 21:23:11.785335 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:11.785341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:11.785448 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:11.785588 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:11.786836 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:11.788026 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:11.789892 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:11.790563 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:11.790846 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:11.792047 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:11.792054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:11.792059 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:11.793039 (+ 980us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:11.793137 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:11.793168 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:11.793307 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:11.793434 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 21:23:11.793559 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 21:23:11.801882 17940 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:11.803848 17940 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:11.812533 17940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e1efa5eef7b43c58d13b2c0c1f84db4: Generated new TSK 0
I20260926 21:23:11.813481 17940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:23:11.823108 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17858
I20260926 21:23:11.824530 17920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:11.917407 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17858
2026-09-26T21:23:11Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2017 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:12 dist-test-slave-4d5t krb5kdc[17959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:12 dist-test-slave-4d5t krb5kdc[17959](info): set up 2 sockets
Sep 26 21:23:12 dist-test-slave-4d5t krb5kdc[17959](info): commencing operation
krb5kdc: starting...
W20260926 21:23:13.054639 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:13 dist-test-slave-4d5t krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:13.276479 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36097
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:36097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:13.623203 17972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:13.623543 17972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:13.623710 17972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:13.631076 17972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:13.631194 17972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:13.631289 17972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:13.631398 17972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:13.631477 17972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:13.641752 17972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:36097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:13.643173 17972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:13.644503 17972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:13.657088 17980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:13.657222 17977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:13.657222 17978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:13.658494 17972 server_base.cc:1034] running on GCE node
I20260926 21:23:13.659399 17972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:13.661053 17972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:13.662307 17972 hybrid_clock.cc:648] HybridClock initialized: now 1790457793662239 us; error 46 us; skew 500 ppm
I20260926 21:23:13.662760 17972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:13 dist-test-slave-4d5t krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457793, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:13.666348 17972 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:13.668032 17972 webserver.cc:460] Webserver started at http://127.14.142.62:43391/ using document root <none> and password file <none>
I20260926 21:23:13.668859 17972 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:13.668972 17972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:13.669320 17972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:13.672484 17972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "04c028530140414cb1ec68db8f31d576"
format_stamp: "Formatted at 2026-09-26 21:23:13 on dist-test-slave-4d5t"
server_key: "1c499da1b8927bdb0af45ab64d6f4b7c"
server_key_iv: "52bf6a2db6e66d38b0971b05969232b6"
server_key_version: "encryptionkey@0"
I20260926 21:23:13.673408 17972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "04c028530140414cb1ec68db8f31d576"
format_stamp: "Formatted at 2026-09-26 21:23:13 on dist-test-slave-4d5t"
server_key: "1c499da1b8927bdb0af45ab64d6f4b7c"
server_key_iv: "52bf6a2db6e66d38b0971b05969232b6"
server_key_version: "encryptionkey@0"
I20260926 21:23:13.680028 17972 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 21:23:13.684340 17987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:13.686156 17972 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:23:13.686389 17972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "04c028530140414cb1ec68db8f31d576"
format_stamp: "Formatted at 2026-09-26 21:23:13 on dist-test-slave-4d5t"
server_key: "1c499da1b8927bdb0af45ab64d6f4b7c"
server_key_iv: "52bf6a2db6e66d38b0971b05969232b6"
server_key_version: "encryptionkey@0"
I20260926 21:23:13.686630 17972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:13.710110 17972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:13.716028 17972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:13.716393 17972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:13.736656 18038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:36097 every 8 connection(s)
I20260926 21:23:13.736663 17972 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:36097
I20260926 21:23:13.738066 17972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:13.739645 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 17972
I20260926 21:23:13.739889 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:13.740712 14904 external_mini_cluster.cc:1428] Setting key 3663b78b92b851f120de709c67456156
I20260926 21:23:13.743052 18039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 21:23:13 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:13.757515 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576: Bootstrap starting.
I20260926 21:23:13.761745 18039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:13.762998 18039 log.cc:824] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:13.766423 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576: No bootstrap required, opened a new log
I20260926 21:23:13.767915 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:13.743576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59840 (local address 127.14.142.62:36097)
0926 21:23:13.744709 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:13.744721 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:13.744766 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:23:13.746364 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:13.746386 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:13.746821 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:13.747209 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:13.749163 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:13.750570 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:13.752474 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:13.753451 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:13.753721 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:13.758529 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:13.758538 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:13.758555 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:13.758599 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:13.763201 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:13.764092 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:13.764104 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:13.764118 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:13.764344 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:13.764840 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:13.764845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:13.764849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:13.765441 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:13.765792 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:13.766332 (+ 540us) server_negotiation.cc:299] Negotiation successful
0926 21:23:13.766814 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":533,"threads_started":1}
I20260926 21:23:13.776042 18039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c028530140414cb1ec68db8f31d576" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36097 } }
I20260926 21:23:13.776818 18039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:13.777009 18039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c028530140414cb1ec68db8f31d576, State: Initialized, Role: FOLLOWER
I20260926 21:23:13.777884 18039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c028530140414cb1ec68db8f31d576" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36097 } }
I20260926 21:23:13.778268 18039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:13.778457 18039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:13.778649 18039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:13.781142 18039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c028530140414cb1ec68db8f31d576" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36097 } }
I20260926 21:23:13.781746 18039 leader_election.cc:302] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04c028530140414cb1ec68db8f31d576; no voters:
I20260926 21:23:13.782416 18039 leader_election.cc:288] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:13.782738 18044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:13.783864 18044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [term 1 LEADER]: Becoming Leader. State: Replica: 04c028530140414cb1ec68db8f31d576, State: Running, Role: LEADER
I20260926 21:23:13.784780 18044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c028530140414cb1ec68db8f31d576" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36097 } }
I20260926 21:23:13.785200 18039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:13.789722 18046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04c028530140414cb1ec68db8f31d576. Latest consensus state: current_term: 1 leader_uuid: "04c028530140414cb1ec68db8f31d576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c028530140414cb1ec68db8f31d576" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36097 } } }
I20260926 21:23:13.790304 18046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:13.791064 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04c028530140414cb1ec68db8f31d576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04c028530140414cb1ec68db8f31d576" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36097 } } }
I20260926 21:23:13.791975 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:13.799129 18051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:13.805150 18051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:13.816809 18051 catalog_manager.cc:1347] Generated new cluster ID: 2311049523a84a58ac29dc9690d9839e
I20260926 21:23:13.817037 18051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:13.832891 18051 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:13.834386 18051 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:13.843801 18051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04c028530140414cb1ec68db8f31d576: Generated new TSK 0
I20260926 21:23:13.844769 18051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:23:13.901420 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36097
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:14.259825 18065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:14.260138 18065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:14.260303 18065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:14.269419 18065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:14.269552 18065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:14.269776 18065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:14.281016 18065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:14.282480 18065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:14.283917 18065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:14.299757 18073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:14.299929 18071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:14.300112 18070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:14.300863 18065 server_base.cc:1034] running on GCE node
I20260926 21:23:14.301614 18065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:14.302857 18065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:14.304315 18065 hybrid_clock.cc:648] HybridClock initialized: now 1790457794304226 us; error 62 us; skew 500 ppm
I20260926 21:23:14.304884 18065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:14 dist-test-slave-4d5t krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457794, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:14.309015 18065 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:23:14.311087 18065 webserver.cc:460] Webserver started at http://127.14.142.1:34751/ using document root <none> and password file <none>
I20260926 21:23:14.311889 18065 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:14.312054 18065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:14.312392 18065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:14.315773 18065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "95118b86c52748b2972e39f313515f62"
format_stamp: "Formatted at 2026-09-26 21:23:14 on dist-test-slave-4d5t"
server_key: "275cec7bb15db531987078a787e4ec6b"
server_key_iv: "43d43cb43ab886e41e7ea26d4b60c583"
server_key_version: "encryptionkey@0"
I20260926 21:23:14.316748 18065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "95118b86c52748b2972e39f313515f62"
format_stamp: "Formatted at 2026-09-26 21:23:14 on dist-test-slave-4d5t"
server_key: "275cec7bb15db531987078a787e4ec6b"
server_key_iv: "43d43cb43ab886e41e7ea26d4b60c583"
server_key_version: "encryptionkey@0"
I20260926 21:23:14.322513 18065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:14.326751 18080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:14.328223 18065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:23:14.328454 18065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "95118b86c52748b2972e39f313515f62"
format_stamp: "Formatted at 2026-09-26 21:23:14 on dist-test-slave-4d5t"
server_key: "275cec7bb15db531987078a787e4ec6b"
server_key_iv: "43d43cb43ab886e41e7ea26d4b60c583"
server_key_version: "encryptionkey@0"
I20260926 21:23:14.328795 18065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:14.354313 18065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:14.359316 18065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:14.359678 18065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:14.360895 18065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:14.362457 18065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:14.362540 18065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:14.362636 18065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:14.362686 18065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:14.400627 18065 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46561
I20260926 21:23:14.400657 18192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46561 every 8 connection(s)
I20260926 21:23:14.402119 18065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:14.406038 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18065
I20260926 21:23:14.406283 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:14.407111 14904 external_mini_cluster.cc:1428] Setting key 0d76c6519b779f1bb25a528dadcec641
Sep 26 21:23:14 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457794, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:14.424986 18195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:14.404886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:49287 (local address 127.14.142.62:36097)
0926 21:23:14.405374 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:14.405378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:14.406411 (+ 1033us) server_negotiation.cc:407] Connection header received
0926 21:23:14.408767 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:14.408772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:14.408864 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:14.408995 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:14.411146 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.411963 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.414542 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.414986 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.415152 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:14.419012 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:14.419017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:14.419021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:14.419057 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:14.421724 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:14.422535 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:14.422547 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:14.422550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:14.422678 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:14.423146 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:14.423150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:14.423152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:14.423426 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:14.423597 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:14.424336 (+ 739us) server_negotiation.cc:299] Negotiation successful
0926 21:23:14.424608 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":275,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:23:14.426071 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:14.405316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36097 (local address 127.14.142.1:49287)
0926 21:23:14.406199 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:14.406252 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:14.408446 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:14.409290 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:14.409302 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:14.410062 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:14.410922 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:14.410959 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.412125 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.412133 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:14.414351 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:14.414379 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.415263 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.415270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:14.415506 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:14.416235 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:14.416260 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:14.418760 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:14.421891 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:14.421901 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:14.421930 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:14.422395 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:14.422798 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:14.422803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:14.422807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:14.423015 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:14.423572 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:14.423581 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:14.423948 (+ 367us) client_negotiation.cc:769] Sending connection context
0926 21:23:14.424223 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 21:23:14.424817 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":303,"threads_started":1}
I20260926 21:23:14.427946 18193 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36097
I20260926 21:23:14.428283 18193 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:14.429102 18193 heartbeater.cc:511] Master 127.14.142.62:36097 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:23:14.431691 18004 ts_manager.cc:194] Registered new tserver with Master: 95118b86c52748b2972e39f313515f62 (127.14.142.1:46561)
I20260926 21:23:14.433743 18004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:49287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:23:14.455971 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36097
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:14.793399 18199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:14.793721 18199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:14.793853 18199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:14.801417 18199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:14.801568 18199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:14.801781 18199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:14.812134 18199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:14.813588 18199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:14.814989 18199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:14.829167 18204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:14.829208 18205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:14.829047 18207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:14.830291 18199 server_base.cc:1034] running on GCE node
I20260926 21:23:14.830902 18199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:14.831980 18199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:14.833215 18199 hybrid_clock.cc:648] HybridClock initialized: now 1790457794833167 us; error 30 us; skew 500 ppm
I20260926 21:23:14.833578 18199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:14 dist-test-slave-4d5t krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457794, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:14.842796 18199 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:23:14.844604 18199 webserver.cc:460] Webserver started at http://127.14.142.2:37853/ using document root <none> and password file <none>
I20260926 21:23:14.845314 18199 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:14.845423 18199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:14.845687 18199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:14.848834 18199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "8b35ac3581194515b590208c58e579a4"
format_stamp: "Formatted at 2026-09-26 21:23:14 on dist-test-slave-4d5t"
server_key: "3bb914831c307e559734a323f2b2b017"
server_key_iv: "a2da6afc0c3759d15dc77c9da2ca7d22"
server_key_version: "encryptionkey@0"
I20260926 21:23:14.849650 18199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "8b35ac3581194515b590208c58e579a4"
format_stamp: "Formatted at 2026-09-26 21:23:14 on dist-test-slave-4d5t"
server_key: "3bb914831c307e559734a323f2b2b017"
server_key_iv: "a2da6afc0c3759d15dc77c9da2ca7d22"
server_key_version: "encryptionkey@0"
I20260926 21:23:14.855129 18199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:14.858917 18214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:14.860447 18199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:23:14.860709 18199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "8b35ac3581194515b590208c58e579a4"
format_stamp: "Formatted at 2026-09-26 21:23:14 on dist-test-slave-4d5t"
server_key: "3bb914831c307e559734a323f2b2b017"
server_key_iv: "a2da6afc0c3759d15dc77c9da2ca7d22"
server_key_version: "encryptionkey@0"
I20260926 21:23:14.860934 18199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:14.891371 18199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:14.896159 18199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:14.896533 18199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:14.897672 18199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:14.899531 18199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:14.899626 18199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:14.899796 18199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:14.899881 18199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:14.940464 18199 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:41349
I20260926 21:23:14.940579 18326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:41349 every 8 connection(s)
I20260926 21:23:14.941860 18199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:14.950075 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18199
I20260926 21:23:14.950292 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:14.950737 14904 external_mini_cluster.cc:1428] Setting key 11933ea9361a547fbd1e8909d8989a3d
Sep 26 21:23:14 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457794, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:14.967336 18329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:14.944658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:57655 (local address 127.14.142.62:36097)
0926 21:23:14.945172 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:14.945177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:14.946269 (+ 1092us) server_negotiation.cc:407] Connection header received
0926 21:23:14.948426 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:14.948433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:14.948530 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:14.948694 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:14.950315 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.951153 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.953104 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.953554 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.953695 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:14.958253 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:14.958258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:14.958262 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:14.958299 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:14.963153 (+ 4854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:14.964163 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:14.964173 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:14.964180 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:14.964375 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:14.965157 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:14.965165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:14.965171 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:14.965617 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:14.965797 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:14.966569 (+ 772us) server_negotiation.cc:299] Negotiation successful
0926 21:23:14.966833 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":280,"threads_started":1}
I20260926 21:23:14.968024 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:14.945034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36097 (local address 127.14.142.2:57655)
0926 21:23:14.945948 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:14.946043 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:14.948051 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:14.948967 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:14.948976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:14.949505 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:14.950094 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:14.950111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.951320 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.951333 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:14.952955 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:14.952968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:14.953791 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:14.953800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:14.954047 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:14.954922 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:14.954959 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:14.957972 (+ 3013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:14.963398 (+ 5426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:14.963407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:14.963432 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:14.963985 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:14.964605 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:14.964609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:14.964612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:14.964989 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:14.965748 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:14.965755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:14.966177 (+ 422us) client_negotiation.cc:769] Sending connection context
0926 21:23:14.966418 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 21:23:14.966842 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":299,"threads_started":1}
I20260926 21:23:14.969636 18327 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36097
I20260926 21:23:14.969998 18327 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:14.971064 18327 heartbeater.cc:511] Master 127.14.142.62:36097 requested a full tablet report, sending...
I20260926 21:23:14.973126 18004 ts_manager.cc:194] Registered new tserver with Master: 8b35ac3581194515b590208c58e579a4 (127.14.142.2:41349)
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:23:14.974265 18004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:57655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:23:15.001645 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36097
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:15.346593 18333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:15.346980 18333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:15.347182 18333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:15.354835 18333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:15.354969 18333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:15.355085 18333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:15.366822 18333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:15.368510 18333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:15.369958 18333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:15.385455 18339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:15.385653 18341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:15.385671 18338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:15.386541 18333 server_base.cc:1034] running on GCE node
I20260926 21:23:15.387259 18333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:15.388340 18333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:15.389591 18333 hybrid_clock.cc:648] HybridClock initialized: now 1790457795389600 us; error 139 us; skew 500 ppm
I20260926 21:23:15.389961 18333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:15.393550 18333 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:23:15.395224 18333 webserver.cc:460] Webserver started at http://127.14.142.3:33877/ using document root <none> and password file <none>
I20260926 21:23:15.395957 18333 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:15.396107 18333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:15.396430 18333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:15.399482 18333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "77a030942de84d739ea0ac2450840954"
format_stamp: "Formatted at 2026-09-26 21:23:15 on dist-test-slave-4d5t"
server_key: "e1837759bde168fe48eb570c99d15ebb"
server_key_iv: "9724b8b7cf33805c2f8750406efdb764"
server_key_version: "encryptionkey@0"
I20260926 21:23:15.400288 18333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "77a030942de84d739ea0ac2450840954"
format_stamp: "Formatted at 2026-09-26 21:23:15 on dist-test-slave-4d5t"
server_key: "e1837759bde168fe48eb570c99d15ebb"
server_key_iv: "9724b8b7cf33805c2f8750406efdb764"
server_key_version: "encryptionkey@0"
I20260926 21:23:15.405839 18333 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 21:23:15.409333 18348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:15.410738 18333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:15.410941 18333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "77a030942de84d739ea0ac2450840954"
format_stamp: "Formatted at 2026-09-26 21:23:15 on dist-test-slave-4d5t"
server_key: "e1837759bde168fe48eb570c99d15ebb"
server_key_iv: "9724b8b7cf33805c2f8750406efdb764"
server_key_version: "encryptionkey@0"
I20260926 21:23:15.411158 18333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:15.437287 18333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:15.437533 18193 heartbeater.cc:503] Master 127.14.142.62:36097 was elected leader, sending a full tablet report...
I20260926 21:23:15.442747 18333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:15.443054 18333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:15.444175 18333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:15.445526 18333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:15.445600 18333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:15.445685 18333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:15.445732 18333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:15.481828 18333 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:39311
I20260926 21:23:15.481917 18460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:39311 every 8 connection(s)
I20260926 21:23:15.483166 18333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:15.484745 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18333
I20260926 21:23:15.484970 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:15.485522 14904 external_mini_cluster.cc:1428] Setting key cba95d7397cb42d462c17d26b3fb7491
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:15.507300 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:15.486330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:44231 (local address 127.14.142.62:36097)
0926 21:23:15.486857 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:15.486862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:15.488342 (+ 1480us) server_negotiation.cc:407] Connection header received
0926 21:23:15.490672 (+ 2330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:15.490678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:15.490787 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:15.490935 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:15.492679 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.493543 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.495250 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.495702 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.495856 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:15.500260 (+ 4404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:15.500267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:15.500272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:15.500315 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:15.503957 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.504876 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.504881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.504883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.505016 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.505441 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.505445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.505447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.505746 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:15.505901 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:15.506682 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 21:23:15.506849 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":269,"threads_started":1}
I20260926 21:23:15.508096 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:15.487027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36097 (local address 127.14.142.3:44231)
0926 21:23:15.488166 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:15.488219 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:15.490390 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:15.491219 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:15.491228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:15.491773 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:15.492481 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:15.492501 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.493706 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.493713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:15.495071 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:15.495107 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.495963 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.495971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:15.496219 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:15.497004 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:15.497029 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:15.500003 (+ 2974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:15.504126 (+ 4123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:15.504134 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:15.504150 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:15.504718 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:15.505122 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:15.505125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:15.505128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:15.505310 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:15.505923 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:15.505930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:15.506301 (+ 371us) client_negotiation.cc:769] Sending connection context
0926 21:23:15.506546 (+ 245us) client_negotiation.cc:240] Negotiation successful
0926 21:23:15.506964 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":532,"threads_started":1}
I20260926 21:23:15.509740 18461 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36097
I20260926 21:23:15.510090 18461 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:15.510856 18461 heartbeater.cc:511] Master 127.14.142.62:36097 requested a full tablet report, sending...
I20260926 21:23:15.512425 18004 ts_manager.cc:194] Registered new tserver with Master: 77a030942de84d739ea0ac2450840954 (127.14.142.3:39311)
I20260926 21:23:15.513554 18004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:44231
I20260926 21:23:15.515760 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:15.573173 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:15.554650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.14.142.62:36097)
0926 21:23:15.554969 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:15.554976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:15.555276 (+ 300us) server_negotiation.cc:407] Connection header received
0926 21:23:15.555434 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:15.555438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:15.555525 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:15.555833 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:15.557533 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.558387 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.560143 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.560707 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.560828 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:15.567261 (+ 6433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:15.567267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:15.567270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:15.567316 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:15.570238 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.571079 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.571083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.571086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.571240 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.571632 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.571637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.571639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.572021 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:15.572264 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:15.572494 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 21:23:15.572804 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
W20260926 21:23:15.573758 18003 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37348
I20260926 21:23:15.575899 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18065
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:15.595117 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:15.577394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.14.142.1:46561)
0926 21:23:15.578146 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:15.578153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:15.578180 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:15.578239 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:15.578246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:15.578460 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:15.578613 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:15.579801 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.581442 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.582716 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.583415 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.583547 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:15.589057 (+ 5510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:15.589071 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:15.589087 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:15.589128 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:15.592190 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.593138 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.593145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.593148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.593292 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.593692 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.593695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.593697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.594084 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:15.594325 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:15.594589 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 21:23:15.594800 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":403,"threads_started":1}
W20260926 21:23:15.596444 18188 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59662
W20260926 21:23:15.597690 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18065: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:23:15.597924 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18065
I20260926 21:23:15.615576 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18199
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:23:15.633931 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:15.616450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.14.142.2:41349)
0926 21:23:15.617020 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:15.617027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:15.617055 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:23:15.617118 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:15.617125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:15.617339 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:15.617479 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:15.618719 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.619925 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.621353 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.621947 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.622058 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:15.627327 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:15.627343 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:15.627359 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:15.627398 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:15.630601 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.631406 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.631414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.631416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.631550 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.631919 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.631929 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.631931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.632303 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:15.632532 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:15.633216 (+ 684us) server_negotiation.cc:299] Negotiation successful
0926 21:23:15.633465 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":310,"threads_started":1}
W20260926 21:23:15.635277 18322 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48702
W20260926 21:23:15.636454 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18199: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:23:15.636709 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18199
I20260926 21:23:15.652792 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18333
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[17959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457795, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:23:15.673950 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:15.653964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.14.142.3:39311)
0926 21:23:15.654618 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:15.654626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:15.654656 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:23:15.654718 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:15.654724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:15.654958 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:15.655133 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:15.656654 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.657743 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.659511 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:15.660295 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:15.660476 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:15.665937 (+ 5461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:15.665956 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:15.665974 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:15.666041 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:15.670031 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.671299 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.671309 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.671312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.671507 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:15.671946 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:15.671951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:15.671957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:15.672411 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:15.672836 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:15.673162 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 21:23:15.673517 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":349,"threads_started":1}
W20260926 21:23:15.675709 18456 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42686
W20260926 21:23:15.676997 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18333: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:23:15.677186 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18333
I20260926 21:23:15.694113 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 17972
I20260926 21:23:15.695367 18034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:15.807096 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 17972
2026-09-26T21:23:15Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3896 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[18479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[18479](info): set up 2 sockets
Sep 26 21:23:15 dist-test-slave-4d5t krb5kdc[18479](info): commencing operation
krb5kdc: starting...
W20260926 21:23:16.934000 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:17 dist-test-slave-4d5t krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:17.161477 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36485
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:36485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:17.493588 18492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:17.493924 18492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:17.494067 18492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:17.501555 18492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:17.501679 18492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:17.501729 18492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:17.501833 18492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:17.501914 18492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:17.512799 18492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:36485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:17.514210 18492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:17.515583 18492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:17.528246 18497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:17.528250 18500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:17.528194 18498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:17.529183 18492 server_base.cc:1034] running on GCE node
I20260926 21:23:17.530014 18492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:17.531627 18492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:17.532907 18492 hybrid_clock.cc:648] HybridClock initialized: now 1790457797532861 us; error 62 us; skew 500 ppm
I20260926 21:23:17.533392 18492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:17 dist-test-slave-4d5t krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457797, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:17.536976 18492 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:17.538694 18492 webserver.cc:460] Webserver started at http://127.14.142.62:43797/ using document root <none> and password file <none>
I20260926 21:23:17.539446 18492 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:17.539598 18492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:17.539928 18492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:17.543088 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "122fc5d49bb44a42ae6cbc0346fa0b23"
format_stamp: "Formatted at 2026-09-26 21:23:17 on dist-test-slave-4d5t"
server_key: "a35468fa716a85dbb7c28bb7b3336016"
server_key_iv: "d96acad13c4116447c36395578c309e7"
server_key_version: "encryptionkey@0"
I20260926 21:23:17.543955 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "122fc5d49bb44a42ae6cbc0346fa0b23"
format_stamp: "Formatted at 2026-09-26 21:23:17 on dist-test-slave-4d5t"
server_key: "a35468fa716a85dbb7c28bb7b3336016"
server_key_iv: "d96acad13c4116447c36395578c309e7"
server_key_version: "encryptionkey@0"
I20260926 21:23:17.549628 18492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 21:23:17.553732 18507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:17.555197 18492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:23:17.555424 18492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "122fc5d49bb44a42ae6cbc0346fa0b23"
format_stamp: "Formatted at 2026-09-26 21:23:17 on dist-test-slave-4d5t"
server_key: "a35468fa716a85dbb7c28bb7b3336016"
server_key_iv: "d96acad13c4116447c36395578c309e7"
server_key_version: "encryptionkey@0"
I20260926 21:23:17.555646 18492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:17.572439 18492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:17.577373 18492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:17.577673 18492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:17.597674 18492 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:36485
I20260926 21:23:17.597712 18558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:36485 every 8 connection(s)
I20260926 21:23:17.599097 18492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:17.603394 18559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:17.604943 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18492
I20260926 21:23:17.605155 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:17.605974 14904 external_mini_cluster.cc:1428] Setting key 897e42d05b40aff19de8a19d99194a3c
I20260926 21:23:17.617275 18559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23: Bootstrap starting.
Sep 26 21:23:17 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:17.620766 18559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:17.622027 18559 log.cc:824] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:17.625597 18559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23: No bootstrap required, opened a new log
I20260926 21:23:17.628880 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:17.608171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.14.142.62:36485)
0926 21:23:17.609030 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:17.609043 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:17.609083 (+ 40us) server_negotiation.cc:407] Connection header received
0926 21:23:17.610681 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:17.610705 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:17.611103 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:17.611521 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:17.613104 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:17.614437 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:17.615691 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:17.616377 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:17.616589 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:17.620939 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:17.620950 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:17.620967 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:17.621009 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:17.624462 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:17.625448 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:17.625455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:17.625462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:17.625641 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:17.626043 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:17.626052 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:17.626061 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:17.626570 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:17.626888 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:17.627348 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 21:23:17.627782 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":410,"threads_started":1}
I20260926 21:23:17.633962 18559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36485 } }
I20260926 21:23:17.634392 18559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:17.634506 18559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122fc5d49bb44a42ae6cbc0346fa0b23, State: Initialized, Role: FOLLOWER
I20260926 21:23:17.635106 18559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36485 } }
I20260926 21:23:17.635372 18559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:17.635520 18559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:17.635695 18559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:17.638425 18559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36485 } }
I20260926 21:23:17.639006 18559 leader_election.cc:302] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 122fc5d49bb44a42ae6cbc0346fa0b23; no voters:
I20260926 21:23:17.639753 18559 leader_election.cc:288] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:17.640091 18564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:17.641027 18564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [term 1 LEADER]: Becoming Leader. State: Replica: 122fc5d49bb44a42ae6cbc0346fa0b23, State: Running, Role: LEADER
I20260926 21:23:17.641767 18564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36485 } }
I20260926 21:23:17.642155 18559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:17.644745 18566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 122fc5d49bb44a42ae6cbc0346fa0b23. Latest consensus state: current_term: 1 leader_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36485 } } }
I20260926 21:23:17.645222 18566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:17.645539 18565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122fc5d49bb44a42ae6cbc0346fa0b23" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36485 } } }
I20260926 21:23:17.645902 18565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:17.648948 18570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:17.656018 18570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:17.666623 18570 catalog_manager.cc:1347] Generated new cluster ID: 0fad771803b747158e236a98a1da8165
I20260926 21:23:17.666764 18570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:17.709391 18570 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:17.710830 18570 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:17.725317 18570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 122fc5d49bb44a42ae6cbc0346fa0b23: Generated new TSK 0
I20260926 21:23:17.726163 18570 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:23:17.784956 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36485
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:18.144608 18585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:18.144927 18585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:18.145119 18585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:18.153012 18585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:18.153138 18585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:18.153343 18585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:18.163689 18585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:18.165186 18585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:18.166525 18585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:18.181625 18590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:18.181664 18591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:18.181576 18593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:18.182521 18585 server_base.cc:1034] running on GCE node
I20260926 21:23:18.183243 18585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:18.184403 18585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:18.185712 18585 hybrid_clock.cc:648] HybridClock initialized: now 1790457798185572 us; error 130 us; skew 500 ppm
I20260926 21:23:18.186201 18585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:18 dist-test-slave-4d5t krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:18.190814 18585 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:23:18.192955 18585 webserver.cc:460] Webserver started at http://127.14.142.1:38011/ using document root <none> and password file <none>
I20260926 21:23:18.193805 18585 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:18.193956 18585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:18.194413 18585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:18.198467 18585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "f755bbac41214ea6816d311451a7347a"
format_stamp: "Formatted at 2026-09-26 21:23:18 on dist-test-slave-4d5t"
server_key: "4f6e8805d629f60bbad80b44a27b5ccf"
server_key_iv: "5927859b745ea0a0914479face7b9df3"
server_key_version: "encryptionkey@0"
I20260926 21:23:18.199486 18585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "f755bbac41214ea6816d311451a7347a"
format_stamp: "Formatted at 2026-09-26 21:23:18 on dist-test-slave-4d5t"
server_key: "4f6e8805d629f60bbad80b44a27b5ccf"
server_key_iv: "5927859b745ea0a0914479face7b9df3"
server_key_version: "encryptionkey@0"
I20260926 21:23:18.206146 18585 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 21:23:18.209877 18600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:18.211310 18585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 21:23:18.211504 18585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "f755bbac41214ea6816d311451a7347a"
format_stamp: "Formatted at 2026-09-26 21:23:18 on dist-test-slave-4d5t"
server_key: "4f6e8805d629f60bbad80b44a27b5ccf"
server_key_iv: "5927859b745ea0a0914479face7b9df3"
server_key_version: "encryptionkey@0"
I20260926 21:23:18.211706 18585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:18.231220 18585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:18.236959 18585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:18.237449 18585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:18.239095 18585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:18.241216 18585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:18.241340 18585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:18.241482 18585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:18.241544 18585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:18.286367 18585 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:45961
I20260926 21:23:18.286379 18712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:45961 every 8 connection(s)
I20260926 21:23:18.288259 18585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:18.291484 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18585
I20260926 21:23:18.291733 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:18.292640 14904 external_mini_cluster.cc:1428] Setting key 6544a22ffc03dc2190f2216e885176e5
Sep 26 21:23:18 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:18.316223 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:18.291148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51085 (local address 127.14.142.62:36485)
0926 21:23:18.291668 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:18.291674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:18.292829 (+ 1155us) server_negotiation.cc:407] Connection header received
0926 21:23:18.295506 (+ 2677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:18.295513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:18.295610 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:18.295741 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:18.298075 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.298822 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.301510 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.302326 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.302528 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:18.307523 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:18.307533 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:18.307541 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:18.307589 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:18.311585 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:18.313053 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:18.313058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:18.313061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:18.313208 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:18.313923 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:18.313927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:18.313929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:18.314235 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:18.314374 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:18.315570 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0926 21:23:18.315872 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":267,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:23:18.317869 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:18.291641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36485 (local address 127.14.142.1:51085)
0926 21:23:18.292606 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:18.292667 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:18.295179 (+ 2512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:18.296060 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:18.296074 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:18.296988 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:18.297858 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:18.297887 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.298994 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.299003 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:18.301274 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:18.301295 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.302623 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.302629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:18.303082 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:18.304190 (+ 1108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:18.304231 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:18.307216 (+ 2985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:18.311833 (+ 4617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:18.311851 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:18.311871 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:18.312842 (+ 971us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:18.313355 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:18.313364 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:18.313370 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:18.313712 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:18.314397 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:18.314411 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:18.315044 (+ 633us) client_negotiation.cc:769] Sending connection context
0926 21:23:18.315463 (+ 419us) client_negotiation.cc:240] Negotiation successful
0926 21:23:18.316236 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":331,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:23:18.320070 18713 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36485
I20260926 21:23:18.320452 18713 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:18.321379 18713 heartbeater.cc:511] Master 127.14.142.62:36485 requested a full tablet report, sending...
I20260926 21:23:18.324131 18524 ts_manager.cc:194] Registered new tserver with Master: f755bbac41214ea6816d311451a7347a (127.14.142.1:45961)
I20260926 21:23:18.326617 18524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:51085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:23:18.349426 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36485
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:18.689541 18719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:18.689836 18719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:18.689934 18719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:18.697269 18719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:18.697396 18719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:18.697518 18719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:18.707536 18719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:18.709299 18719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:18.711210 18719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:18.726603 18725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:18.726691 18727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:18.726585 18724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:18.728543 18719 server_base.cc:1034] running on GCE node
I20260926 21:23:18.729305 18719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:18.730494 18719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:18.731751 18719 hybrid_clock.cc:648] HybridClock initialized: now 1790457798731685 us; error 48 us; skew 500 ppm
I20260926 21:23:18.732136 18719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:18 dist-test-slave-4d5t krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:18.735728 18719 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:23:18.737469 18719 webserver.cc:460] Webserver started at http://127.14.142.2:43137/ using document root <none> and password file <none>
I20260926 21:23:18.738129 18719 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:18.738227 18719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:18.738483 18719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:18.741560 18719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "68c4ea4a707d4589a88db1db9a3e5416"
format_stamp: "Formatted at 2026-09-26 21:23:18 on dist-test-slave-4d5t"
server_key: "940b55e1fc0e38ffc73397f894b4b1ea"
server_key_iv: "c45ec9770aa9c03c896394d2231edaf4"
server_key_version: "encryptionkey@0"
I20260926 21:23:18.742365 18719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "68c4ea4a707d4589a88db1db9a3e5416"
format_stamp: "Formatted at 2026-09-26 21:23:18 on dist-test-slave-4d5t"
server_key: "940b55e1fc0e38ffc73397f894b4b1ea"
server_key_iv: "c45ec9770aa9c03c896394d2231edaf4"
server_key_version: "encryptionkey@0"
I20260926 21:23:18.748080 18719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 21:23:18.752009 18734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:18.753674 18719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:23:18.753863 18719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "68c4ea4a707d4589a88db1db9a3e5416"
format_stamp: "Formatted at 2026-09-26 21:23:18 on dist-test-slave-4d5t"
server_key: "940b55e1fc0e38ffc73397f894b4b1ea"
server_key_iv: "c45ec9770aa9c03c896394d2231edaf4"
server_key_version: "encryptionkey@0"
I20260926 21:23:18.754086 18719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:18.790895 18719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:18.796041 18719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:18.796427 18719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:18.797551 18719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:18.799067 18719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:18.799185 18719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:18.799356 18719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:18.799437 18719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:18.839385 18719 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:39901
I20260926 21:23:18.839569 18846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:39901 every 8 connection(s)
I20260926 21:23:18.840803 18719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:18.842554 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18719
I20260926 21:23:18.842731 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:18.843499 14904 external_mini_cluster.cc:1428] Setting key be217fcbd62412d5ed19bdd2be9e9bc0
Sep 26 21:23:18 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457798, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:23:18.869706 18849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:18.844019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:49767 (local address 127.14.142.62:36485)
0926 21:23:18.844691 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:18.844700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:18.845524 (+ 824us) server_negotiation.cc:407] Connection header received
0926 21:23:18.848704 (+ 3180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:18.848713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:18.848821 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:18.848931 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:18.851337 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.852459 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.854313 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.855447 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.855707 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:18.860958 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:18.860970 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:18.860979 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:18.861046 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:18.865546 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:18.866824 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:18.866829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:18.866832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:18.867012 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:18.867680 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:18.867684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:18.867686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:18.868008 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:18.868125 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:18.869246 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0926 21:23:18.869435 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":276,"threads_started":1}
I20260926 21:23:18.870808 18850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:18.844544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36485 (local address 127.14.142.2:49767)
0926 21:23:18.845355 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:18.845394 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:18.848256 (+ 2862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:18.849240 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:18.849255 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:18.850171 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:18.851093 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:18.851123 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.852685 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.852694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:18.854146 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:18.854160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:18.855859 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:18.855865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:18.856098 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:18.857133 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:18.857175 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:18.860502 (+ 3327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:18.865819 (+ 5317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:18.865833 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:18.865859 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:18.866645 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:18.867188 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:18.867195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:18.867200 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:18.867539 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:18.868183 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:18.868196 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:18.868773 (+ 577us) client_negotiation.cc:769] Sending connection context
0926 21:23:18.869166 (+ 393us) client_negotiation.cc:240] Negotiation successful
0926 21:23:18.869625 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":344,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
I20260926 21:23:18.872455 18847 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36485
I20260926 21:23:18.872815 18847 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:18.873616 18847 heartbeater.cc:511] Master 127.14.142.62:36485 requested a full tablet report, sending...
I20260926 21:23:18.875298 18524 ts_manager.cc:194] Registered new tserver with Master: 68c4ea4a707d4589a88db1db9a3e5416 (127.14.142.2:39901)
I20260926 21:23:18.876610 18524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:49767
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:23:18.900192 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36485
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:19.251592 18853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:19.251991 18853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:19.252287 18853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:19.261749 18853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:19.261937 18853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:19.262135 18853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:19.272703 18853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:19.274127 18853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:19.275471 18853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:19.289482 18861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:19.289711 18858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:19.290202 18859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:19.290740 18853 server_base.cc:1034] running on GCE node
I20260926 21:23:19.291363 18853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:19.292456 18853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:19.295898 18853 hybrid_clock.cc:648] HybridClock initialized: now 1790457799295822 us; error 130 us; skew 500 ppm
I20260926 21:23:19.296335 18853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:19.300799 18853 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:23:19.302541 18853 webserver.cc:460] Webserver started at http://127.14.142.3:40105/ using document root <none> and password file <none>
I20260926 21:23:19.303242 18853 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:19.303357 18853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:19.303634 18853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:19.306689 18853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "ccb7dcafb6f84175a1cf2303e9a32b2b"
format_stamp: "Formatted at 2026-09-26 21:23:19 on dist-test-slave-4d5t"
server_key: "ba2b86fd61ba7522507727cf265153d7"
server_key_iv: "9a9039069749009f53a5fa8a2a4304cc"
server_key_version: "encryptionkey@0"
I20260926 21:23:19.307442 18853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "ccb7dcafb6f84175a1cf2303e9a32b2b"
format_stamp: "Formatted at 2026-09-26 21:23:19 on dist-test-slave-4d5t"
server_key: "ba2b86fd61ba7522507727cf265153d7"
server_key_iv: "9a9039069749009f53a5fa8a2a4304cc"
server_key_version: "encryptionkey@0"
I20260926 21:23:19.312924 18853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260926 21:23:19.317493 18868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:19.319084 18853 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:19.319244 18853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "ccb7dcafb6f84175a1cf2303e9a32b2b"
format_stamp: "Formatted at 2026-09-26 21:23:19 on dist-test-slave-4d5t"
server_key: "ba2b86fd61ba7522507727cf265153d7"
server_key_iv: "9a9039069749009f53a5fa8a2a4304cc"
server_key_version: "encryptionkey@0"
I20260926 21:23:19.319451 18853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:19.330518 18713 heartbeater.cc:503] Master 127.14.142.62:36485 was elected leader, sending a full tablet report...
I20260926 21:23:19.338660 18853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:19.344489 18853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:19.344859 18853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:19.346189 18853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:19.347667 18853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:19.347795 18853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:19.347980 18853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:19.348052 18853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:19.385365 18853 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:42151
I20260926 21:23:19.385432 18980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:42151 every 8 connection(s)
I20260926 21:23:19.386685 18853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:19.391781 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 18853
I20260926 21:23:19.391973 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:19.392437 14904 external_mini_cluster.cc:1428] Setting key 9001acd74b905f087a5d0de50c7b79fd
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:19.408993 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.389298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:34045 (local address 127.14.142.62:36485)
0926 21:23:19.389751 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:19.389756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:19.390559 (+ 803us) server_negotiation.cc:407] Connection header received
0926 21:23:19.392508 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:19.392513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:19.392654 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:19.392774 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:19.394648 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.395600 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.397430 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.398024 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.398202 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.402405 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:19.402411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:19.402415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:19.402463 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:19.405578 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.406493 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.406499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.406505 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.406633 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.407097 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.407103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.407107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.407395 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:19.407568 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:19.408374 (+ 806us) server_negotiation.cc:299] Negotiation successful
0926 21:23:19.408668 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":256,"threads_started":1}
I20260926 21:23:19.409931 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.389706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36485 (local address 127.14.142.3:34045)
0926 21:23:19.390400 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:19.390436 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:19.392238 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:19.393073 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:19.393087 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:19.393664 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:19.394417 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:19.394437 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.395754 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.395760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:19.397068 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:19.397080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.398316 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.398324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:19.398546 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.399279 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:19.399307 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:19.402128 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:19.405749 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:19.405757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:19.405773 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:19.406361 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:19.406745 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:19.406750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:19.406754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:19.406984 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:19.407546 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:19.407553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:19.407991 (+ 438us) client_negotiation.cc:769] Sending connection context
0926 21:23:19.408241 (+ 250us) client_negotiation.cc:240] Negotiation successful
0926 21:23:19.408735 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":288,"threads_started":1}
I20260926 21:23:19.411389 18981 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36485
I20260926 21:23:19.411736 18981 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:19.412537 18981 heartbeater.cc:511] Master 127.14.142.62:36485 requested a full tablet report, sending...
I20260926 21:23:19.414446 18524 ts_manager.cc:194] Registered new tserver with Master: ccb7dcafb6f84175a1cf2303e9a32b2b (127.14.142.3:42151)
I20260926 21:23:19.415409 18524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:34045
I20260926 21:23:19.420706 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:19.474249 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.457219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.14.142.1:45961)
0926 21:23:19.457924 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:19.457931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:19.457964 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:23:19.458032 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:19.458038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:19.458309 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:19.458529 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:19.460256 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.461356 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.462635 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.463217 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.463528 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.468050 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:19.468066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:19.468083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:19.468131 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:19.471137 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.471971 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.471981 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.471987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.472125 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.472521 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.472526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.472530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.472955 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:19.473286 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:19.473586 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 21:23:19.473865 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":359,"threads_started":1}
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:19.495420 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.479601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.14.142.62:36485)
0926 21:23:19.479871 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:19.479877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:19.479973 (+ 96us) server_negotiation.cc:407] Connection header received
0926 21:23:19.480178 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:19.480183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:19.480267 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:19.480359 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:19.482265 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.483027 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.484645 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.485055 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.485407 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.489861 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:19.489866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:19.489869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:19.489916 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:19.492527 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.493339 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.493344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.493348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.493492 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.493886 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.493892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.493896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.494250 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:19.494529 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:19.494843 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 21:23:19.495099 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260926 21:23:19.496075 18524 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53700
I20260926 21:23:19.497249 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18585
I20260926 21:23:19.512362 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.498402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.14.142.1:45961)
0926 21:23:19.498601 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:19.498605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:19.498798 (+ 193us) server_negotiation.cc:407] Connection header received
0926 21:23:19.498899 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:19.498904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:19.499066 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:19.499198 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:19.500789 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.501545 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.502896 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.503361 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.503491 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.506101 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:19.506108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:19.506112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:19.506165 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:19.509596 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.510335 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.510341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.510343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.510510 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.510863 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.510868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.510870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.511168 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:19.511366 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:19.511771 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 21:23:19.512048 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:23:19.513170 18708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:19.599702 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18585
I20260926 21:23:19.623754 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18719
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:23:19.645051 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.625324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50606 (local address 127.14.142.2:39901)
0926 21:23:19.626014 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:19.626022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:19.626052 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:23:19.626124 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:19.626131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:19.626401 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:19.626627 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:19.628636 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.629697 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.631606 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.632190 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.632352 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.638031 (+ 5679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:19.638056 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:19.638074 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:19.638121 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:19.641530 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.642365 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.642373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.642375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.642521 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.642910 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.642915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.642918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.643510 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:19.643908 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:19.644267 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 21:23:19.644611 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":406,"threads_started":1}
I20260926 21:23:19.646428 18842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:19.733218 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18719
I20260926 21:23:19.750581 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18853
Sep 26 21:23:19 dist-test-slave-4d5t krb5kdc[18479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:23:19.769843 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:19.751655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.14.142.3:42151)
0926 21:23:19.752297 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:19.752304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:19.752334 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:23:19.752400 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:19.752406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:19.752716 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:19.752853 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:19.754202 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.755513 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.757396 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:19.758058 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:19.758194 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:19.763124 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:19.763139 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:19.763157 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:19.763201 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:19.766841 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.767663 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.767671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.767673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.767803 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:19.768206 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:19.768209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:19.768211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:19.768740 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:19.769071 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:19.769342 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 21:23:19.769550 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":328,"threads_started":1}
I20260926 21:23:19.771335 18976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:19.860351 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18853
I20260926 21:23:19.876828 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 18492
I20260926 21:23:19.878309 18554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:19.986421 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 18492
2026-09-26T21:23:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4178 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:20 dist-test-slave-4d5t krb5kdc[19001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:20 dist-test-slave-4d5t krb5kdc[19001](info): set up 2 sockets
Sep 26 21:23:20 dist-test-slave-4d5t krb5kdc[19001](info): commencing operation
krb5kdc: starting...
W20260926 21:23:21.146070 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:21 dist-test-slave-4d5t krb5kdc[19001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:21.364017 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40973
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:40973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:21.708843 19014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:21.709179 19014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:21.709331 19014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:21.716962 19014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:21.717087 19014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:21.717167 19014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:21.717275 19014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:21.717363 19014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:21.727635 19014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:40973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:21.729117 19014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:21.730526 19014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:21.743444 19019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:21.743498 19020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:21.743534 19022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:21.744702 19014 server_base.cc:1034] running on GCE node
I20260926 21:23:21.745710 19014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:21.747342 19014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:21.748652 19014 hybrid_clock.cc:648] HybridClock initialized: now 1790457801748550 us; error 37 us; skew 500 ppm
I20260926 21:23:21.749070 19014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:21 dist-test-slave-4d5t krb5kdc[19001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457801, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:21.752990 19014 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:21.754766 19014 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:21.754966 19014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:21.755339 19014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:21.758713 19014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "f2cdb2f9932b4c1bb1178339d622a75b"
format_stamp: "Formatted at 2026-09-26 21:23:21 on dist-test-slave-4d5t"
server_key: "3e5a5aa39fc8e581732d30f36a086f13"
server_key_iv: "9a9e9ef642c64ec1dedd2761a7700dad"
server_key_version: "encryptionkey@0"
I20260926 21:23:21.759575 19014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "f2cdb2f9932b4c1bb1178339d622a75b"
format_stamp: "Formatted at 2026-09-26 21:23:21 on dist-test-slave-4d5t"
server_key: "3e5a5aa39fc8e581732d30f36a086f13"
server_key_iv: "9a9e9ef642c64ec1dedd2761a7700dad"
server_key_version: "encryptionkey@0"
I20260926 21:23:21.765102 19014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:21.769130 19028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:21.771263 19014 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 21:23:21.771682 19014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "f2cdb2f9932b4c1bb1178339d622a75b"
format_stamp: "Formatted at 2026-09-26 21:23:21 on dist-test-slave-4d5t"
server_key: "3e5a5aa39fc8e581732d30f36a086f13"
server_key_iv: "9a9e9ef642c64ec1dedd2761a7700dad"
server_key_version: "encryptionkey@0"
I20260926 21:23:21.772147 19014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:21.791499 19014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:21.798418 19014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:21.798779 19014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:21.821698 19014 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:40973
I20260926 21:23:21.821681 19079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:40973 every 8 connection(s)
I20260926 21:23:21.822674 19014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:21.826674 19080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:21.827653 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19014
I20260926 21:23:21.827883 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:21.828622 14904 external_mini_cluster.cc:1428] Setting key 14707089b5e2cfab59071ad940224539
I20260926 21:23:21.840818 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b: Bootstrap starting.
I20260926 21:23:21.844775 19080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:21.845847 19080 log.cc:824] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b: Log is configured to *not* fsync() on all Append() calls
Sep 26 21:23:21 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:21.849062 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b: No bootstrap required, opened a new log
I20260926 21:23:21.855716 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:21.831406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.14.142.62:40973)
0926 21:23:21.832646 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:21.832665 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:21.832732 (+ 67us) server_negotiation.cc:407] Connection header received
0926 21:23:21.835196 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:21.835230 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:21.835797 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:21.836288 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:21.838283 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:21.840231 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:21.842288 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:21.843427 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:21.843650 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:21.848067 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:21.848076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:21.848092 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:21.848132 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:21.851290 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:21.852092 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:21.852100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:21.852106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:21.852281 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:21.852714 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:21.852717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:21.852719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:21.853292 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:21.853646 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:21.854109 (+ 463us) server_negotiation.cc:299] Negotiation successful
0926 21:23:21.854556 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":916,"thread_start_us":464,"threads_started":1}
I20260926 21:23:21.857524 19080 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40973 } }
I20260926 21:23:21.858034 19080 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:21.858147 19080 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2cdb2f9932b4c1bb1178339d622a75b, State: Initialized, Role: FOLLOWER
I20260926 21:23:21.858793 19080 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40973 } }
I20260926 21:23:21.859058 19080 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:21.859200 19080 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:21.859325 19080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:21.861562 19080 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40973 } }
I20260926 21:23:21.862025 19080 leader_election.cc:302] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2cdb2f9932b4c1bb1178339d622a75b; no voters:
I20260926 21:23:21.862599 19080 leader_election.cc:288] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:21.862763 19085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:21.863523 19085 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [term 1 LEADER]: Becoming Leader. State: Replica: f2cdb2f9932b4c1bb1178339d622a75b, State: Running, Role: LEADER
I20260926 21:23:21.864228 19085 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40973 } }
I20260926 21:23:21.865053 19080 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:21.867265 19087 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2cdb2f9932b4c1bb1178339d622a75b. Latest consensus state: current_term: 1 leader_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40973 } } }
I20260926 21:23:21.867388 19086 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cdb2f9932b4c1bb1178339d622a75b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40973 } } }
I20260926 21:23:21.867673 19087 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:21.867700 19086 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:21.873118 19090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:21.878507 19090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:21.889424 19090 catalog_manager.cc:1347] Generated new cluster ID: 6e7a5c3e822c496dbcaa8b3794c2642c
I20260926 21:23:21.889556 19090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:21.916700 19090 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:21.918196 19090 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:21.928519 19090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b: Generated new TSK 0
I20260926 21:23:21.929803 19090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:23:21.993495 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40973
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:22.342648 19106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:22.342968 19106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:22.343087 19106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:22.351534 19106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:22.351733 19106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:22.351961 19106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:22.362296 19106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:22.363667 19106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:22.365013 19106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:22.379922 19114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:22.380002 19111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:22.379983 19112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:22.381429 19106 server_base.cc:1034] running on GCE node
I20260926 21:23:22.382119 19106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:22.383153 19106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:22.384377 19106 hybrid_clock.cc:648] HybridClock initialized: now 1790457802384301 us; error 63 us; skew 500 ppm
I20260926 21:23:22.384778 19106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:22 dist-test-slave-4d5t krb5kdc[19001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457802, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:22.388034 19106 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:23:22.389240 19106 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:22.389333 19106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:22.389580 19106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:22.392448 19106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "ca2ae481176f46eb82250731b9e8454a"
format_stamp: "Formatted at 2026-09-26 21:23:22 on dist-test-slave-4d5t"
server_key: "ae7adc99b2d00ab3251ce427156ef13e"
server_key_iv: "c99111364ebbce682a57b6529796a87c"
server_key_version: "encryptionkey@0"
I20260926 21:23:22.393318 19106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "ca2ae481176f46eb82250731b9e8454a"
format_stamp: "Formatted at 2026-09-26 21:23:22 on dist-test-slave-4d5t"
server_key: "ae7adc99b2d00ab3251ce427156ef13e"
server_key_iv: "c99111364ebbce682a57b6529796a87c"
server_key_version: "encryptionkey@0"
I20260926 21:23:22.398798 19106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:22.402835 19120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:22.404361 19106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:23:22.404656 19106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "ca2ae481176f46eb82250731b9e8454a"
format_stamp: "Formatted at 2026-09-26 21:23:22 on dist-test-slave-4d5t"
server_key: "ae7adc99b2d00ab3251ce427156ef13e"
server_key_iv: "c99111364ebbce682a57b6529796a87c"
server_key_version: "encryptionkey@0"
I20260926 21:23:22.404930 19106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:22.443499 19106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:22.448153 19106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:22.448475 19106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:22.449560 19106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:22.450974 19106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:22.451103 19106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:22.451238 19106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:22.451328 19106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:22.489297 19106 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:33743
I20260926 21:23:22.489388 19232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:33743 every 8 connection(s)
I20260926 21:23:22.490221 19106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:22.497071 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19106
I20260926 21:23:22.497280 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:22.497936 14904 external_mini_cluster.cc:1428] Setting key 8450f6b398fa20990f36ce0d3f44db14
Sep 26 21:23:22 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457802, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:22.515173 19235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:22.492895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:53801 (local address 127.14.142.62:40973)
0926 21:23:22.493408 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:22.493413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:22.494327 (+ 914us) server_negotiation.cc:407] Connection header received
0926 21:23:22.496127 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:22.496133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:22.496228 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:22.496350 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:22.498053 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:22.498942 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:22.500844 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:22.501663 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:22.501846 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:22.506313 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:22.506325 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:22.506331 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:22.506384 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:22.510319 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:22.511751 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:22.511758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:22.511761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:22.511966 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:22.512727 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:22.512733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:22.512736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:22.513092 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:22.513229 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:22.514433 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0926 21:23:22.514828 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":287,"threads_started":1}
I20260926 21:23:22.516743 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:22.493348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40973 (local address 127.14.142.1:53801)
0926 21:23:22.494119 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:22.494161 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:22.495845 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:22.496634 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:22.496642 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:22.497219 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:22.497856 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:22.497874 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:22.499101 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:22.499105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:22.500503 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:22.500520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:22.502071 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:22.502079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:22.502380 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:22.503322 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:22.503351 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:22.505948 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:22.510524 (+ 4576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:22.510540 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:22.510564 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:22.511453 (+ 889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:22.512137 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:22.512146 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:22.512153 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:22.512463 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:22.513253 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:22.513265 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:22.513819 (+ 554us) client_negotiation.cc:769] Sending connection context
0926 21:23:22.514191 (+ 372us) client_negotiation.cc:240] Negotiation successful
0926 21:23:22.514944 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":328,"threads_started":1}
I20260926 21:23:22.518698 19233 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40973
I20260926 21:23:22.519037 19233 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:22.519765 19233 heartbeater.cc:511] Master 127.14.142.62:40973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:23:22.522883 19045 ts_manager.cc:194] Registered new tserver with Master: ca2ae481176f46eb82250731b9e8454a (127.14.142.1:33743)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:23:22.524726 19045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:53801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:23:22.551914 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40973
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:22.897245 19239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:22.897588 19239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:22.897760 19239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:22.906016 19239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:22.906178 19239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:22.906359 19239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:22.916514 19239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:22.917922 19239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:22.919286 19239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:22.932667 19245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:22.932813 19244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:22.932801 19247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:22.938289 19239 server_base.cc:1034] running on GCE node
I20260926 21:23:22.939064 19239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:22.940217 19239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:22.941488 19239 hybrid_clock.cc:648] HybridClock initialized: now 1790457802941446 us; error 101 us; skew 500 ppm
I20260926 21:23:22.941946 19239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:22 dist-test-slave-4d5t krb5kdc[19001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457802, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:22.946077 19239 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:23:22.947448 19239 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:22.947589 19239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:22.947942 19239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:22.951093 19239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "79a14cfa74374b85b0eaafb2684ad0d2"
format_stamp: "Formatted at 2026-09-26 21:23:22 on dist-test-slave-4d5t"
server_key: "6c48f0257fde693469c8d08441d7166e"
server_key_iv: "92ec6dbd6e56844582e69df66d12ab85"
server_key_version: "encryptionkey@0"
I20260926 21:23:22.951948 19239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "79a14cfa74374b85b0eaafb2684ad0d2"
format_stamp: "Formatted at 2026-09-26 21:23:22 on dist-test-slave-4d5t"
server_key: "6c48f0257fde693469c8d08441d7166e"
server_key_iv: "92ec6dbd6e56844582e69df66d12ab85"
server_key_version: "encryptionkey@0"
I20260926 21:23:22.957415 19239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 21:23:22.961475 19253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:22.963675 19239 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:22.963925 19239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "79a14cfa74374b85b0eaafb2684ad0d2"
format_stamp: "Formatted at 2026-09-26 21:23:22 on dist-test-slave-4d5t"
server_key: "6c48f0257fde693469c8d08441d7166e"
server_key_iv: "92ec6dbd6e56844582e69df66d12ab85"
server_key_version: "encryptionkey@0"
I20260926 21:23:22.964176 19239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:22.990550 19239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:22.995352 19239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:22.995672 19239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:22.996815 19239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:22.998283 19239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:22.998428 19239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:22.998562 19239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:22.998659 19239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:23.035799 19239 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:41695
I20260926 21:23:23.035895 19365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:41695 every 8 connection(s)
I20260926 21:23:23.036764 19239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:23.044428 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19239
I20260926 21:23:23.044648 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:23.045272 14904 external_mini_cluster.cc:1428] Setting key 4662da0f55f4431e43e2faae6bfd3c44
Sep 26 21:23:23 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457802, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:23.064249 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.039424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:49107 (local address 127.14.142.62:40973)
0926 21:23:23.039888 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.039893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.041027 (+ 1134us) server_negotiation.cc:407] Connection header received
0926 21:23:23.043262 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.043269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.043385 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.043564 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:23.046005 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.047038 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.049494 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.049984 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.050179 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.056744 (+ 6565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:23.056753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:23.056758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:23.056810 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:23.060402 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.061683 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.061689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.061693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.061841 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.062316 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.062322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.062325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.062629 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:23.062794 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.063607 (+ 813us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.063856 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":227,"threads_started":1}
I20260926 21:23:23.065572 19369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.039958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40973 (local address 127.14.142.2:49107)
0926 21:23:23.040847 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.040893 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.042820 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.043781 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.043791 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.044728 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:23.045672 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.045702 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.047259 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.047264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.049277 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.049298 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.050401 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.050415 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.050785 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.051700 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:23.051735 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:23.056322 (+ 4587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:23.060628 (+ 4306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:23.060640 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:23.060662 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:23.061542 (+ 880us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:23.061966 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:23.061973 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:23.061977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:23.062182 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:23.062770 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:23.062779 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:23.063181 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.063500 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.064098 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":364,"threads_started":1}
I20260926 21:23:23.067701 19366 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40973
I20260926 21:23:23.068212 19366 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:23:23.069427 19366 heartbeater.cc:511] Master 127.14.142.62:40973 requested a full tablet report, sending...
I20260926 21:23:23.071580 19045 ts_manager.cc:194] Registered new tserver with Master: 79a14cfa74374b85b0eaafb2684ad0d2 (127.14.142.2:41695)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
I20260926 21:23:23.072851 19045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:49107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:23:23.101977 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40973
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:43031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:23.441069 19372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:23.441401 19372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:23.441514 19372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:23.450870 19372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:23.451002 19372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:23.451125 19372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:23.461426 19372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:43031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:23.462821 19372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:23.464178 19372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:23.478367 19377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:23.478415 19380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:23.478479 19378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:23.479871 19372 server_base.cc:1034] running on GCE node
I20260926 21:23:23.480703 19372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:23.481832 19372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:23.483181 19372 hybrid_clock.cc:648] HybridClock initialized: now 1790457803483115 us; error 70 us; skew 500 ppm
I20260926 21:23:23.483587 19372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:23 dist-test-slave-4d5t krb5kdc[19001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:23.487095 19372 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:23:23.488503 19372 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:23.488672 19372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:23.489084 19372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:23.492267 19372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "c4b13b5ba259453cb679b0abc6c89921"
format_stamp: "Formatted at 2026-09-26 21:23:23 on dist-test-slave-4d5t"
server_key: "de64d51f3cc0ba503ed2577f1d49fd58"
server_key_iv: "1731115f3419882d033ded6a095b3515"
server_key_version: "encryptionkey@0"
I20260926 21:23:23.493371 19372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "c4b13b5ba259453cb679b0abc6c89921"
format_stamp: "Formatted at 2026-09-26 21:23:23 on dist-test-slave-4d5t"
server_key: "de64d51f3cc0ba503ed2577f1d49fd58"
server_key_iv: "1731115f3419882d033ded6a095b3515"
server_key_version: "encryptionkey@0"
I20260926 21:23:23.499006 19372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:23.503450 19386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:23.505240 19372 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:23:23.505420 19372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "c4b13b5ba259453cb679b0abc6c89921"
format_stamp: "Formatted at 2026-09-26 21:23:23 on dist-test-slave-4d5t"
server_key: "de64d51f3cc0ba503ed2577f1d49fd58"
server_key_iv: "1731115f3419882d033ded6a095b3515"
server_key_version: "encryptionkey@0"
I20260926 21:23:23.505684 19372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:23.530014 19233 heartbeater.cc:503] Master 127.14.142.62:40973 was elected leader, sending a full tablet report...
I20260926 21:23:23.531177 19372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:23.536273 19372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:23.536662 19372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:23.537847 19372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:23.539511 19372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:23.539608 19372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:23.539798 19372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:23.539872 19372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:23.581568 19372 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:45407
I20260926 21:23:23.581655 19498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:45407 every 8 connection(s)
I20260926 21:23:23.582474 19372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:23.585861 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19372
I20260926 21:23:23.586114 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:23.586577 14904 external_mini_cluster.cc:1428] Setting key f44eff3516ea907a14f87d553763d772
Sep 26 21:23:23 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457803, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:23.607740 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.585691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:45181 (local address 127.14.142.62:40973)
0926 21:23:23.586242 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.586249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.587662 (+ 1413us) server_negotiation.cc:407] Connection header received
0926 21:23:23.590147 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.590154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.590255 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.590360 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:23.592314 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.593256 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.595238 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.595734 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.595878 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.600440 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:23.600448 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:23.600453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:23.600503 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:23.604178 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.605199 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.605204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.605207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.605372 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.605852 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.605856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.605858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.606194 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:23.606326 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.607224 (+ 898us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.607426 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":281,"threads_started":1}
I20260926 21:23:23.609783 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.586562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40973 (local address 127.14.142.3:45181)
0926 21:23:23.587457 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.587526 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.589765 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.590785 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.590798 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.591437 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:23.592096 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.592115 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.593434 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.593438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.594880 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.594894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.596111 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.596118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.596428 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.597492 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:23.597520 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:23.600112 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:23.604384 (+ 4272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:23.604395 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:23.604415 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:23.605064 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:23.605503 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:23.605507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:23.605510 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:23.605712 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:23.606358 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:23.606364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:23.606766 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.607186 (+ 420us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.608010 (+ 824us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":406,"threads_started":1}
I20260926 21:23:23.611758 19499 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40973
I20260926 21:23:23.612126 19499 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:23.612926 19499 heartbeater.cc:511] Master 127.14.142.62:40973 requested a full tablet report, sending...
I20260926 21:23:23.614862 19045 ts_manager.cc:194] Registered new tserver with Master: c4b13b5ba259453cb679b0abc6c89921 (127.14.142.3:45407)
I20260926 21:23:23.615998 19045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:45181
I20260926 21:23:23.616616 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:23:23.636670 19501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.622479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.14.142.62:40973)
0926 21:23:23.622711 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.622716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.623519 (+ 803us) server_negotiation.cc:407] Connection header received
0926 21:23:23.623781 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.623794 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.623872 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.624019 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:23.625845 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.626760 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.628125 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.628622 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.628780 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.631264 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:23.631269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:23.631273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:23.631310 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:23.633910 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.634620 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.634626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.634632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.634779 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.635163 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.635170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.635173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.635514 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:23.635786 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.636087 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.636322 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 21:23:23.641268 19045 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:23.648416 19045 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:23.674206 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.662780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.14.142.1:33743)
0926 21:23:23.664385 (+ 1605us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.664399 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.664435 (+ 36us) server_negotiation.cc:407] Connection header received
0926 21:23:23.664584 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.664593 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.664880 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.665768 (+ 888us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:23.666137 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.667891 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.671256 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.672874 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.673035 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.673295 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.673636 (+ 341us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.673828 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1409,"thread_start_us":1240,"threads_started":1}
I20260926 21:23:23.674304 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.663634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:45407 (local address 127.0.0.1:48758)
0926 21:23:23.666285 (+ 2651us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.666304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.666447 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.667097 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.667104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.667131 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:23.667625 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.667639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.669689 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.669694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.671350 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.671364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.673466 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.673471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.673644 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.673675 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.673764 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.673886 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2522,"mutex_wait_us":7,"thread_start_us":880,"threads_started":1}
I20260926 21:23:23.674345 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.662421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:33743 (local address 127.0.0.1:37114)
0926 21:23:23.663258 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.663324 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.663595 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.665175 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.665186 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.665225 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:23.665649 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.665664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.668075 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.668082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.671056 (+ 2974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.671072 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.673114 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.673121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.673335 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.673406 (+ 71us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.673565 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.673796 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":2,"thread_start_us":334,"threads_started":1}
I20260926 21:23:23.675419 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.664208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.14.142.3:45407)
0926 21:23:23.665458 (+ 1250us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.665465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.666418 (+ 953us) server_negotiation.cc:407] Connection header received
0926 21:23:23.666604 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.666611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.666811 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.666996 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:23.667882 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.669554 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.671548 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.673056 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.673167 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.673359 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.674870 (+ 1511us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.675050 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":1298,"threads_started":1}
I20260926 21:23:23.675727 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.663294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41695 (local address 127.0.0.1:49674)
0926 21:23:23.664675 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.664695 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.664812 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.666837 (+ 2025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.666843 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.666871 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:23.667305 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.667324 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.669369 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.669375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.672633 (+ 3258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.672648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.674755 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.674761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.674948 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.674998 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.675140 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.675287 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235,"mutex_wait_us":27,"thread_start_us":2777,"threads_started":1}
I20260926 21:23:23.676705 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.663931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.14.142.2:41695)
0926 21:23:23.666181 (+ 2250us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.666189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.666216 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:23.666309 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.666314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.666626 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.666816 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:23.667509 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.669133 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.672847 (+ 3714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.674500 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.674696 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.674971 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.675184 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.675376 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2097,"thread_start_us":1959,"threads_started":1}
I20260926 21:23:23.678988 19168 tablet_service.cc:1467] Processing CreateTablet for tablet c36fd6b5dde84b23a465a78e2300bfeb (DEFAULT_TABLE table=test-table [id=7b3e84f352d44948a4e4260cc7e18b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:23.679313 19434 tablet_service.cc:1467] Processing CreateTablet for tablet c36fd6b5dde84b23a465a78e2300bfeb (DEFAULT_TABLE table=test-table [id=7b3e84f352d44948a4e4260cc7e18b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:23.680619 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36fd6b5dde84b23a465a78e2300bfeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:23.680620 19434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36fd6b5dde84b23a465a78e2300bfeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:23.680449 19301 tablet_service.cc:1467] Processing CreateTablet for tablet c36fd6b5dde84b23a465a78e2300bfeb (DEFAULT_TABLE table=test-table [id=7b3e84f352d44948a4e4260cc7e18b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:23.681798 19301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36fd6b5dde84b23a465a78e2300bfeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:23.692886 19518 tablet_bootstrap.cc:492] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Bootstrap starting.
I20260926 21:23:23.695899 19519 tablet_bootstrap.cc:492] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Bootstrap starting.
I20260926 21:23:23.698823 19518 tablet_bootstrap.cc:654] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:23.699810 19519 tablet_bootstrap.cc:654] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:23.699831 19520 tablet_bootstrap.cc:492] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Bootstrap starting.
I20260926 21:23:23.700860 19518 log.cc:824] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:23.701318 19519 log.cc:824] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:23.703176 19520 tablet_bootstrap.cc:654] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:23.704231 19520 log.cc:824] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:23.705410 19519 tablet_bootstrap.cc:492] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: No bootstrap required, opened a new log
I20260926 21:23:23.705904 19519 ts_tablet_manager.cc:1397] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260926 21:23:23.710917 19518 tablet_bootstrap.cc:492] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: No bootstrap required, opened a new log
I20260926 21:23:23.711071 19520 tablet_bootstrap.cc:492] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: No bootstrap required, opened a new log
I20260926 21:23:23.711305 19518 ts_tablet_manager.cc:1397] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.010s
I20260926 21:23:23.711398 19520 ts_tablet_manager.cc:1397] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260926 21:23:23.718880 19519 raft_consensus.cc:348] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.719431 19519 raft_consensus.cc:374] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:23.719609 19519 raft_consensus.cc:729] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2ae481176f46eb82250731b9e8454a, State: Initialized, Role: FOLLOWER
I20260926 21:23:23.720366 19519 consensus_queue.cc:260] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.721892 19518 raft_consensus.cc:348] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.722301 19518 raft_consensus.cc:374] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:23.722481 19518 raft_consensus.cc:729] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b13b5ba259453cb679b0abc6c89921, State: Initialized, Role: FOLLOWER
I20260926 21:23:23.724009 19519 ts_tablet_manager.cc:1428] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
I20260926 21:23:23.723765 19518 consensus_queue.cc:260] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.723755 19520 raft_consensus.cc:348] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.724284 19520 raft_consensus.cc:374] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:23.724448 19520 raft_consensus.cc:729] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79a14cfa74374b85b0eaafb2684ad0d2, State: Initialized, Role: FOLLOWER
I20260926 21:23:23.725435 19499 heartbeater.cc:503] Master 127.14.142.62:40973 was elected leader, sending a full tablet report...
I20260926 21:23:23.725404 19520 consensus_queue.cc:260] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.726400 19518 ts_tablet_manager.cc:1428] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260926 21:23:23.728227 19366 heartbeater.cc:503] Master 127.14.142.62:40973 was elected leader, sending a full tablet report...
I20260926 21:23:23.730486 19520 ts_tablet_manager.cc:1428] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Time spent starting tablet: real 0.019s user 0.013s sys 0.006s
W20260926 21:23:23.742949 19234 tablet.cc:2369] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:23:23.789026 19367 tablet.cc:2369] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:23.812076 19524 raft_consensus.cc:482] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:23.812417 19524 raft_consensus.cc:504] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.814186 19524 leader_election.cc:288] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4b13b5ba259453cb679b0abc6c89921 (127.14.142.3:45407), 79a14cfa74374b85b0eaafb2684ad0d2 (127.14.142.2:41695)
I20260926 21:23:23.823563 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.814920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51919 (local address 127.14.142.2:41695)
0926 21:23:23.815186 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.815192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.815947 (+ 755us) server_negotiation.cc:407] Connection header received
0926 21:23:23.816191 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.816202 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.816285 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.816401 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:23.817138 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.818050 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.820749 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.822120 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.822279 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.822351 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.823100 (+ 749us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.823251 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260926 21:23:23.823515 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.814564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:54567 (local address 127.14.142.3:45407)
0926 21:23:23.814824 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.814833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.815650 (+ 817us) server_negotiation.cc:407] Connection header received
0926 21:23:23.815874 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.815883 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.815982 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.816089 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:23.816861 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.817736 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.820237 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.821842 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.822034 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.822114 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.822875 (+ 761us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.823158 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 21:23:23.823655 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.814566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:45407 (local address 127.14.142.1:54567)
0926 21:23:23.815466 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.815516 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.815706 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.816179 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.816183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.816212 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:23.816704 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.816716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.817892 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.817898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.819959 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.819975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.822285 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.822292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.822682 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.822712 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.822843 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.823010 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":76,"thread_start_us":441,"threads_started":1}
I20260926 21:23:23.824791 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.814683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41695 (local address 127.14.142.1:51919)
0926 21:23:23.815823 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:23.815842 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:23.815993 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:23.816428 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:23.816432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:23.816453 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:23.816918 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.816935 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.818210 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.818214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.820386 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:23.820415 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.822683 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.822690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:23.822890 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.822926 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:23:23.824200 (+ 1274us) client_negotiation.cc:240] Negotiation successful
0926 21:23:23.824357 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":702,"threads_started":1}
I20260926 21:23:23.824994 19454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36fd6b5dde84b23a465a78e2300bfeb" candidate_uuid: "ca2ae481176f46eb82250731b9e8454a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b13b5ba259453cb679b0abc6c89921" is_pre_election: true
I20260926 21:23:23.825758 19454 raft_consensus.cc:2454] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca2ae481176f46eb82250731b9e8454a in term 0.
I20260926 21:23:23.826059 19321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36fd6b5dde84b23a465a78e2300bfeb" candidate_uuid: "ca2ae481176f46eb82250731b9e8454a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" is_pre_election: true
I20260926 21:23:23.826624 19321 raft_consensus.cc:2454] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca2ae481176f46eb82250731b9e8454a in term 0.
I20260926 21:23:23.826776 19121 leader_election.cc:302] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b13b5ba259453cb679b0abc6c89921, ca2ae481176f46eb82250731b9e8454a; no voters:
I20260926 21:23:23.827478 19524 raft_consensus.cc:2781] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:23.827684 19524 raft_consensus.cc:482] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:23.827841 19524 raft_consensus.cc:3037] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:23.830430 19524 raft_consensus.cc:504] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.831208 19524 leader_election.cc:288] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [CANDIDATE]: Term 1 election: Requested vote from peers c4b13b5ba259453cb679b0abc6c89921 (127.14.142.3:45407), 79a14cfa74374b85b0eaafb2684ad0d2 (127.14.142.2:41695)
I20260926 21:23:23.831890 19321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36fd6b5dde84b23a465a78e2300bfeb" candidate_uuid: "ca2ae481176f46eb82250731b9e8454a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79a14cfa74374b85b0eaafb2684ad0d2"
I20260926 21:23:23.831905 19454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36fd6b5dde84b23a465a78e2300bfeb" candidate_uuid: "ca2ae481176f46eb82250731b9e8454a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4b13b5ba259453cb679b0abc6c89921"
I20260926 21:23:23.832172 19321 raft_consensus.cc:3037] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:23.832340 19454 raft_consensus.cc:3037] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 0 FOLLOWER]: Advancing to term 1
W20260926 21:23:23.834136 19500 tablet.cc:2369] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:23.835292 19321 raft_consensus.cc:2454] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca2ae481176f46eb82250731b9e8454a in term 1.
I20260926 21:23:23.835757 19454 raft_consensus.cc:2454] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca2ae481176f46eb82250731b9e8454a in term 1.
I20260926 21:23:23.835888 19124 leader_election.cc:302] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79a14cfa74374b85b0eaafb2684ad0d2, ca2ae481176f46eb82250731b9e8454a; no voters:
I20260926 21:23:23.836275 19524 raft_consensus.cc:2781] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:23.836875 19524 raft_consensus.cc:686] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 1 LEADER]: Becoming Leader. State: Replica: ca2ae481176f46eb82250731b9e8454a, State: Running, Role: LEADER
I20260926 21:23:23.837463 19524 consensus_queue.cc:237] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } }
I20260926 21:23:23.843273 19045 catalog_manager.cc:5495] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a reported cstate change: term changed from 0 to 1, leader changed from <none> to ca2ae481176f46eb82250731b9e8454a (127.14.142.1). New cstate: current_term: 1 leader_uuid: "ca2ae481176f46eb82250731b9e8454a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca2ae481176f46eb82250731b9e8454a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:23.874264 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.867973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37120 (local address 127.14.142.1:33743)
0926 21:23:23.868224 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.868231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.868383 (+ 152us) server_negotiation.cc:407] Connection header received
0926 21:23:23.868698 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.868703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.868790 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.868997 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:23.869425 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.870203 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.871676 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.872162 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.872336 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.873024 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:23.873597 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:23.873723 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.873880 (+ 157us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.873958 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:23:23.884660 19321 raft_consensus.cc:1261] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 1 FOLLOWER]: Refusing update from remote peer ca2ae481176f46eb82250731b9e8454a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:23.884816 19454 raft_consensus.cc:1261] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 1 FOLLOWER]: Refusing update from remote peer ca2ae481176f46eb82250731b9e8454a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:23.886070 19529 consensus_queue.cc:1035] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4b13b5ba259453cb679b0abc6c89921" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:23.886478 19524 consensus_queue.cc:1035] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79a14cfa74374b85b0eaafb2684ad0d2" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:23.898864 19532 mvcc.cc:205] Tried to move back new op lower bound from 7333715164697145344 to 7333715164522340352. Current Snapshot: MvccSnapshot[applied={T|T < 7333715164697145344}]
I20260926 21:23:23.914762 19043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:23.915109 19043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:23.919567 19043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2cdb2f9932b4c1bb1178339d622a75b: Sending DeleteTablet for 3 replicas of tablet c36fd6b5dde84b23a465a78e2300bfeb
I20260926 21:23:23.921012 19434 tablet_service.cc:1514] Processing DeleteTablet for tablet c36fd6b5dde84b23a465a78e2300bfeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:23 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:48758
I20260926 21:23:23.921023 19301 tablet_service.cc:1514] Processing DeleteTablet for tablet c36fd6b5dde84b23a465a78e2300bfeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:23 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:49674
I20260926 21:23:23.921890 19168 tablet_service.cc:1514] Processing DeleteTablet for tablet c36fd6b5dde84b23a465a78e2300bfeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:23 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:37114
I20260926 21:23:23.923796 19544 tablet_replica.cc:330] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: stopping tablet replica
I20260926 21:23:23.924156 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19106
I20260926 21:23:23.924324 19544 raft_consensus.cc:2229] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:23.925045 19544 raft_consensus.cc:2258] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:23.927695 19544 ts_tablet_manager.cc:1905] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:23.936223 19544 ts_tablet_manager.cc:1918] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:23.938355 19544 log.cc:1196] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/c36fd6b5dde84b23a465a78e2300bfeb
I20260926 21:23:23.939203 19543 tablet_replica.cc:330] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: stopping tablet replica
I20260926 21:23:23.939307 19544 ts_tablet_manager.cc:1939] T c36fd6b5dde84b23a465a78e2300bfeb P ca2ae481176f46eb82250731b9e8454a: Deleting consensus metadata
I20260926 21:23:23.940482 19543 raft_consensus.cc:2229] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:23.941425 19543 raft_consensus.cc:2258] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:23.942873 19542 tablet_replica.cc:330] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: stopping tablet replica
I20260926 21:23:23.943300 19542 raft_consensus.cc:2229] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:23.943825 19542 raft_consensus.cc:2258] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:23.944196 19029 catalog_manager.cc:4841] TS ca2ae481176f46eb82250731b9e8454a (127.14.142.1:33743): tablet c36fd6b5dde84b23a465a78e2300bfeb (table test-table [id=7b3e84f352d44948a4e4260cc7e18b08]) successfully deleted
I20260926 21:23:23.945313 19543 ts_tablet_manager.cc:1905] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 21:23:23 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:23.951056 19542 ts_tablet_manager.cc:1905] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:23.952693 19543 ts_tablet_manager.cc:1918] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:23.952994 19543 log.cc:1196] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/c36fd6b5dde84b23a465a78e2300bfeb
I20260926 21:23:23.953639 19543 ts_tablet_manager.cc:1939] T c36fd6b5dde84b23a465a78e2300bfeb P c4b13b5ba259453cb679b0abc6c89921: Deleting consensus metadata
I20260926 21:23:23.955756 19029 catalog_manager.cc:4841] TS c4b13b5ba259453cb679b0abc6c89921 (127.14.142.3:45407): tablet c36fd6b5dde84b23a465a78e2300bfeb (table test-table [id=7b3e84f352d44948a4e4260cc7e18b08]) successfully deleted
I20260926 21:23:23.958603 19542 ts_tablet_manager.cc:1918] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:23.958806 19542 log.cc:1196] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/c36fd6b5dde84b23a465a78e2300bfeb
I20260926 21:23:23.959329 19542 ts_tablet_manager.cc:1939] T c36fd6b5dde84b23a465a78e2300bfeb P 79a14cfa74374b85b0eaafb2684ad0d2: Deleting consensus metadata
I20260926 21:23:23.961052 19032 catalog_manager.cc:4841] TS 79a14cfa74374b85b0eaafb2684ad0d2 (127.14.142.2:41695): tablet c36fd6b5dde84b23a465a78e2300bfeb (table test-table [id=7b3e84f352d44948a4e4260cc7e18b08]) successfully deleted
I20260926 21:23:23.961410 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:23.925904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37134 (local address 127.14.142.1:33743)
0926 21:23:23.926086 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:23.926092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:23.931640 (+ 5548us) server_negotiation.cc:407] Connection header received
0926 21:23:23.931887 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:23.931893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:23.931996 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:23.932102 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:23.939537 (+ 7435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.940944 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.945625 (+ 4681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:23.946423 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:23.946580 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:23.952653 (+ 6073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:23.952674 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:23.952696 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:23.952768 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:23.957276 (+ 4508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.958346 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.958356 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.958361 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.958564 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:23.959092 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:23.959099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:23.959105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:23.959623 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:23.959952 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:23.960640 (+ 688us) server_negotiation.cc:299] Negotiation successful
0926 21:23:23.960938 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 21:23:23.962491 19228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:24.069864 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19106
I20260926 21:23:24.090878 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19239
Sep 26 21:23:24 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:23:24.106827 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:24.091672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49676 (local address 127.14.142.2:41695)
0926 21:23:24.091885 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:24.091893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:24.091977 (+ 84us) server_negotiation.cc:407] Connection header received
0926 21:23:24.092229 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:24.092234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:24.092317 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:24.092434 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:24.093933 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:24.094692 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:24.095903 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:24.096354 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:24.096499 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:24.100525 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:24.100539 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:24.100600 (+ 61us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:24.100644 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:24.103446 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:24.104249 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:24.104259 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:24.104262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:24.104447 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:24.104989 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:24.105005 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:24.105014 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:24.105458 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:24.105795 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:24.106145 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 21:23:24.106393 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 21:23:24.107664 19361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:24.201195 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19239
I20260926 21:23:24.219901 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19372
Sep 26 21:23:24 dist-test-slave-4d5t krb5kdc[19001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:23:24.237114 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:24.220953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.14.142.3:45407)
0926 21:23:24.221162 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:24.221169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:24.221191 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:23:24.221259 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:24.221263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:24.221346 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:24.221462 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:24.222745 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:24.223502 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:24.225143 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:24.225792 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:24.225912 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:24.230550 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:24.230567 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:24.230585 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:24.230625 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:24.233514 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:24.234233 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:24.234241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:24.234243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:24.234390 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:24.234862 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:24.234865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:24.234868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:24.235274 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:24.235505 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:24.236294 (+ 789us) server_negotiation.cc:299] Negotiation successful
0926 21:23:24.236802 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 21:23:24.237880 19494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:24.338847 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19372
I20260926 21:23:24.356849 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19014
I20260926 21:23:24.358276 19075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:24.471457 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19014
2026-09-26T21:23:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4488 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T21:23:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:24Z Disabled control of system clock
I20260926 21:23:24.546463 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40761
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:39521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:40761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 21:23:24.904484 19553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:24.904884 19553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:24.904994 19553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:24.913894 19553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:24.914036 19553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:24.914074 19553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:24.914124 19553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:24.914165 19553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:24.923811 19553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:40761
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:24.925230 19553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:24.927276 19553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:24.941658 19558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:24.941705 19561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:24.941679 19559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:24.942699 19553 server_base.cc:1034] running on GCE node
I20260926 21:23:24.943619 19553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:24.945315 19553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:24.946574 19553 hybrid_clock.cc:648] HybridClock initialized: now 1790457804946509 us; error 69 us; skew 500 ppm
I20260926 21:23:24.946998 19553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:24.949710 19553 webserver.cc:460] Webserver started at http://127.14.142.62:42277/ using document root <none> and password file <none>
I20260926 21:23:24.950484 19553 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:24.950627 19553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:24.951103 19553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:24.954603 19553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "9275cd05eb0c462ea7ed93098aa2bdbe"
format_stamp: "Formatted at 2026-09-26 21:23:24 on dist-test-slave-4d5t"
server_key: "5ad5b27a2da1032d758487ff7e9aa9b7"
server_key_iv: "273a2e1b80c0ef19e680ca8b998918cf"
server_key_version: "encryptionkey@0"
I20260926 21:23:24.955477 19553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "9275cd05eb0c462ea7ed93098aa2bdbe"
format_stamp: "Formatted at 2026-09-26 21:23:24 on dist-test-slave-4d5t"
server_key: "5ad5b27a2da1032d758487ff7e9aa9b7"
server_key_iv: "273a2e1b80c0ef19e680ca8b998918cf"
server_key_version: "encryptionkey@0"
I20260926 21:23:24.961496 19553 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260926 21:23:24.965441 19567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:24.966907 19553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:24.967118 19553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "9275cd05eb0c462ea7ed93098aa2bdbe"
format_stamp: "Formatted at 2026-09-26 21:23:24 on dist-test-slave-4d5t"
server_key: "5ad5b27a2da1032d758487ff7e9aa9b7"
server_key_iv: "273a2e1b80c0ef19e680ca8b998918cf"
server_key_version: "encryptionkey@0"
I20260926 21:23:24.967363 19553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:24.973536 19553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:24.974381 19553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:24.974689 19553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:24.996538 19553 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:40761
I20260926 21:23:24.996662 19618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:40761 every 8 connection(s)
I20260926 21:23:24.998037 19553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:24.999945 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19553
I20260926 21:23:25.000178 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:25.001047 14904 external_mini_cluster.cc:1428] Setting key 70ff9850078b29075faeadd554b0839d
I20260926 21:23:25.003360 19619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:25.010402 19622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:25.003528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.14.142.62:40761)
0926 21:23:25.004672 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:25.004684 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:25.004731 (+ 47us) server_negotiation.cc:407] Connection header received
0926 21:23:25.006386 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:25.006409 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:25.006884 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:25.007378 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:25.008605 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:25.008619 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:25.008633 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:25.008649 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:25.008731 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:25.008748 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:25.009008 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:25.009167 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 21:23:25.009438 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":403,"threads_started":1}
I20260926 21:23:25.015791 19619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe: Bootstrap starting.
I20260926 21:23:25.019852 19619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:25.020993 19619 log.cc:824] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:25.023895 19619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe: No bootstrap required, opened a new log
I20260926 21:23:25.031064 19619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40761 } }
I20260926 21:23:25.031467 19619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:25.031638 19619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9275cd05eb0c462ea7ed93098aa2bdbe, State: Initialized, Role: FOLLOWER
I20260926 21:23:25.032730 19619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40761 } }
I20260926 21:23:25.033110 19619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:25.033464 19619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:25.033632 19619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:25.035910 19619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40761 } }
I20260926 21:23:25.036468 19619 leader_election.cc:302] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9275cd05eb0c462ea7ed93098aa2bdbe; no voters:
I20260926 21:23:25.037201 19619 leader_election.cc:288] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:25.037361 19624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:25.038117 19624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [term 1 LEADER]: Becoming Leader. State: Replica: 9275cd05eb0c462ea7ed93098aa2bdbe, State: Running, Role: LEADER
I20260926 21:23:25.038712 19624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40761 } }
I20260926 21:23:25.039335 19619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:25.042312 19626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9275cd05eb0c462ea7ed93098aa2bdbe. Latest consensus state: current_term: 1 leader_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40761 } } }
I20260926 21:23:25.042287 19625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9275cd05eb0c462ea7ed93098aa2bdbe" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40761 } } }
I20260926 21:23:25.042742 19626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:25.042739 19625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:25.044446 19632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:25.050184 19632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:25.063380 19632 catalog_manager.cc:1347] Generated new cluster ID: f14a5dd4f16e4aab85068011d40287f0
I20260926 21:23:25.063616 19632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:25.085606 19632 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:25.086138 19632 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:25.096285 19632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe: Generated new TSK 0
I20260926 21:23:25.097373 19632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:23:25.115386 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40761
--builtin_ntp_servers=127.14.142.20:39521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 21:23:25.480341 19643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:25.480788 19643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:25.480919 19643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:25.490224 19643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:25.490350 19643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:25.490470 19643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:25.500213 19643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:25.501576 19643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:25.503362 19643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:25.517501 19651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:25.517584 19648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:25.517580 19649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:25.518337 19643 server_base.cc:1034] running on GCE node
I20260926 21:23:25.519001 19643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:25.520040 19643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:25.521270 19643 hybrid_clock.cc:648] HybridClock initialized: now 1790457805521209 us; error 48 us; skew 500 ppm
I20260926 21:23:25.521572 19643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:25.530642 19643 webserver.cc:460] Webserver started at http://127.14.142.1:37005/ using document root <none> and password file <none>
I20260926 21:23:25.531401 19643 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:25.531497 19643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:25.531769 19643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:25.534801 19643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "f7088e706b424c6ca7af646b7600f519"
format_stamp: "Formatted at 2026-09-26 21:23:25 on dist-test-slave-4d5t"
server_key: "3854f7f59e41114498f506b41662d72c"
server_key_iv: "02edde8f41a09d2b56d4319cec0b9712"
server_key_version: "encryptionkey@0"
I20260926 21:23:25.535593 19643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "f7088e706b424c6ca7af646b7600f519"
format_stamp: "Formatted at 2026-09-26 21:23:25 on dist-test-slave-4d5t"
server_key: "3854f7f59e41114498f506b41662d72c"
server_key_iv: "02edde8f41a09d2b56d4319cec0b9712"
server_key_version: "encryptionkey@0"
I20260926 21:23:25.541770 19643 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 21:23:25.545490 19657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:25.547084 19643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:25.547278 19643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "f7088e706b424c6ca7af646b7600f519"
format_stamp: "Formatted at 2026-09-26 21:23:25 on dist-test-slave-4d5t"
server_key: "3854f7f59e41114498f506b41662d72c"
server_key_iv: "02edde8f41a09d2b56d4319cec0b9712"
server_key_version: "encryptionkey@0"
I20260926 21:23:25.547528 19643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:25.553454 19643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:25.554281 19643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:25.554561 19643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:25.555554 19643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:25.557091 19643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:25.557179 19643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:25.557336 19643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:25.557432 19643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:25.594583 19643 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46005
I20260926 21:23:25.594619 19769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46005 every 8 connection(s)
I20260926 21:23:25.595909 19643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:25.600297 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19643
I20260926 21:23:25.600531 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:25.601362 14904 external_mini_cluster.cc:1428] Setting key 127edddfb46b3b6eb2df2c9e3c48fd06
I20260926 21:23:25.604534 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40761
--builtin_ntp_servers=127.14.142.20:39521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 21:23:25.605281 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:25.598641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51129 (local address 127.14.142.62:40761)
0926 21:23:25.599126 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:25.599135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:25.600502 (+ 1367us) server_negotiation.cc:407] Connection header received
0926 21:23:25.602780 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:25.602785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:25.602849 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:25.602980 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:25.604447 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:25.604452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:25.604456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:25.604474 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:25.604539 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:25.604614 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:25.604812 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:25.604999 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 21:23:25.605053 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":240,"threads_started":1}
I20260926 21:23:25.606612 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:25.599211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40761 (local address 127.14.142.1:51129)
0926 21:23:25.600306 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:25.600353 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:25.602535 (+ 2182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:25.603200 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:25.603208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:25.603297 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:25.604069 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:25.604073 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:25.604122 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:25.604236 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:25.604762 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:25.604775 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:25.604785 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 21:23:25.605062 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 21:23:25.605326 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":299,"threads_started":1}
I20260926 21:23:25.608335 19770 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40761
I20260926 21:23:25.608919 19770 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:25.609833 19770 heartbeater.cc:511] Master 127.14.142.62:40761 requested a full tablet report, sending...
I20260926 21:23:25.612264 19584 ts_manager.cc:194] Registered new tserver with Master: f7088e706b424c6ca7af646b7600f519 (127.14.142.1:46005)
W20260926 21:23:25.942633 19774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:25.942957 19774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:25.943374 19774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:25.951359 19774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:25.951484 19774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:25.951692 19774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:25.961382 19774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:25.962672 19774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:25.964476 19774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:25.978775 19780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:25.978747 19779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:25.979221 19782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:25.980173 19774 server_base.cc:1034] running on GCE node
I20260926 21:23:25.980947 19774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:25.982062 19774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:25.983328 19774 hybrid_clock.cc:648] HybridClock initialized: now 1790457805983290 us; error 34 us; skew 500 ppm
I20260926 21:23:25.983724 19774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:25.986728 19774 webserver.cc:460] Webserver started at http://127.14.142.2:38703/ using document root <none> and password file <none>
I20260926 21:23:25.987501 19774 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:25.987640 19774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:25.987995 19774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:25.991056 19774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "2f6c67a304054d59a3adcb25051d25a0"
format_stamp: "Formatted at 2026-09-26 21:23:25 on dist-test-slave-4d5t"
server_key: "abc43b4e2259ad623df91f2aa6b66ed8"
server_key_iv: "68e3705a4ed96209d1ba5dbe963f7019"
server_key_version: "encryptionkey@0"
I20260926 21:23:25.991926 19774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "2f6c67a304054d59a3adcb25051d25a0"
format_stamp: "Formatted at 2026-09-26 21:23:25 on dist-test-slave-4d5t"
server_key: "abc43b4e2259ad623df91f2aa6b66ed8"
server_key_iv: "68e3705a4ed96209d1ba5dbe963f7019"
server_key_version: "encryptionkey@0"
I20260926 21:23:25.997586 19774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260926 21:23:26.001631 19788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:26.004027 19774 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:26.004246 19774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "2f6c67a304054d59a3adcb25051d25a0"
format_stamp: "Formatted at 2026-09-26 21:23:25 on dist-test-slave-4d5t"
server_key: "abc43b4e2259ad623df91f2aa6b66ed8"
server_key_iv: "68e3705a4ed96209d1ba5dbe963f7019"
server_key_version: "encryptionkey@0"
I20260926 21:23:26.004511 19774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:26.011672 19774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:26.012518 19774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:26.012845 19774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:26.013957 19774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:26.015434 19774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:26.015559 19774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:26.015687 19774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:26.015777 19774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:26.052198 19774 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:38201
I20260926 21:23:26.052340 19900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:38201 every 8 connection(s)
I20260926 21:23:26.053690 19774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:26.058104 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19774
I20260926 21:23:26.058317 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:26.058897 14904 external_mini_cluster.cc:1428] Setting key 81ee11640873874817d335008c9c44f2
I20260926 21:23:26.062788 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40761
--builtin_ntp_servers=127.14.142.20:39521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 21:23:26.064095 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.056749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43323 (local address 127.14.142.62:40761)
0926 21:23:26.057031 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.057041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.058500 (+ 1459us) server_negotiation.cc:407] Connection header received
0926 21:23:26.061267 (+ 2767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.061273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.061335 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.061440 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.063386 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.063390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.063393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.063410 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.063445 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.063458 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.063582 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.063824 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.063875 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 21:23:26.065635 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.057081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40761 (local address 127.14.142.2:43323)
0926 21:23:26.058320 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.058373 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.060830 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.062003 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.062020 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.062170 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.062966 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.062971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.063036 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.063169 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.063582 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.063596 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.063609 (+ 13us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.063930 (+ 321us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.064243 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":335,"threads_started":1}
I20260926 21:23:26.067165 19901 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40761
I20260926 21:23:26.067498 19901 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:26.068215 19901 heartbeater.cc:511] Master 127.14.142.62:40761 requested a full tablet report, sending...
I20260926 21:23:26.070389 19584 ts_manager.cc:194] Registered new tserver with Master: 2f6c67a304054d59a3adcb25051d25a0 (127.14.142.2:38201)
W20260926 21:23:26.405265 19904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:26.405567 19904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:26.405661 19904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:26.413148 19904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:26.413259 19904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:26.413368 19904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:26.422693 19904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:26.424026 19904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:26.425855 19904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:26.439568 19909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:26.439610 19910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:26.440086 19912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:26.441045 19904 server_base.cc:1034] running on GCE node
I20260926 21:23:26.441692 19904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:26.442727 19904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:26.443974 19904 hybrid_clock.cc:648] HybridClock initialized: now 1790457806443920 us; error 50 us; skew 500 ppm
I20260926 21:23:26.444339 19904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:26.446925 19904 webserver.cc:460] Webserver started at http://127.14.142.3:33941/ using document root <none> and password file <none>
I20260926 21:23:26.447674 19904 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:26.447813 19904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:26.448159 19904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:26.451211 19904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "2a438bdd401f4b6cb5d0c291ec300196"
format_stamp: "Formatted at 2026-09-26 21:23:26 on dist-test-slave-4d5t"
server_key: "1010ec93f1349d018f80d0fd7709e188"
server_key_iv: "92c2faa2e1d612bd85dd8fd46d7bfcb7"
server_key_version: "encryptionkey@0"
I20260926 21:23:26.452090 19904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "2a438bdd401f4b6cb5d0c291ec300196"
format_stamp: "Formatted at 2026-09-26 21:23:26 on dist-test-slave-4d5t"
server_key: "1010ec93f1349d018f80d0fd7709e188"
server_key_iv: "92c2faa2e1d612bd85dd8fd46d7bfcb7"
server_key_version: "encryptionkey@0"
I20260926 21:23:26.457496 19904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 21:23:26.461140 19918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:26.462540 19904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 21:23:26.462728 19904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "2a438bdd401f4b6cb5d0c291ec300196"
format_stamp: "Formatted at 2026-09-26 21:23:26 on dist-test-slave-4d5t"
server_key: "1010ec93f1349d018f80d0fd7709e188"
server_key_iv: "92c2faa2e1d612bd85dd8fd46d7bfcb7"
server_key_version: "encryptionkey@0"
I20260926 21:23:26.462971 19904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:26.468946 19904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:26.469805 19904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:26.470101 19904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:26.471110 19904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:26.472777 19904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:26.472898 19904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:26.473045 19904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:26.473135 19904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:26.512132 19904 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:41777
I20260926 21:23:26.512228 20030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:41777 every 8 connection(s)
I20260926 21:23:26.514184 19904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:26.517022 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 19904
I20260926 21:23:26.517256 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:26.517825 14904 external_mini_cluster.cc:1428] Setting key 3a3ac6b9db1eb72ba5aafad75d23cba2
I20260926 21:23:26.524868 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.517583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:37099 (local address 127.14.142.62:40761)
0926 21:23:26.517817 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.517823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.519680 (+ 1857us) server_negotiation.cc:407] Connection header received
0926 21:23:26.522172 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.522178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.522257 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.522397 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.523932 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.523939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.523944 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.523969 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.524018 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.524031 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.524173 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.524404 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.524480 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 21:23:26.526331 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.518261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40761 (local address 127.14.142.3:37099)
0926 21:23:26.519515 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.519555 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.521866 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.522661 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.522671 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.522778 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.523596 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.523599 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.523641 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.523735 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.524147 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.524159 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.524168 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.524505 (+ 337us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.524820 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":381,"threads_started":1}
I20260926 21:23:26.528352 20031 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40761
I20260926 21:23:26.528944 20031 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:26.529711 20031 heartbeater.cc:511] Master 127.14.142.62:40761 requested a full tablet report, sending...
I20260926 21:23:26.531603 19584 ts_manager.cc:194] Registered new tserver with Master: 2a438bdd401f4b6cb5d0c291ec300196 (127.14.142.3:41777)
I20260926 21:23:26.534377 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:23:26.542471 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.539649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47912 (local address 127.14.142.62:40761)
0926 21:23:26.539904 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.539914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.540300 (+ 386us) server_negotiation.cc:407] Connection header received
0926 21:23:26.540529 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.540536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.540640 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.540775 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.541738 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.541744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.541749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.541770 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.541814 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.541827 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.541984 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.542087 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.542154 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 21:23:26.546875 19584 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:26.554255 19584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:26.578472 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.567530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.14.142.2:38201)
0926 21:23:26.574639 (+ 7109us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.574645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.574672 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:26.574767 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.574776 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.575041 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.576680 (+ 1639us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.577526 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.577537 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.577553 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.577574 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.577634 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.577662 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.577798 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.578000 (+ 202us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.578116 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6968,"thread_start_us":6845,"threads_started":1}
I20260926 21:23:26.578873 20045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.567380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38201 (local address 127.0.0.1:34672)
0926 21:23:26.568847 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.568901 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.569281 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.575290 (+ 6009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.575299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.575399 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.576304 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.576308 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.576341 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.576423 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.577760 (+ 1337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.577774 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.577782 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.578329 (+ 547us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.578423 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"thread_start_us":7473,"threads_started":1}
I20260926 21:23:26.579690 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.567887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46005 (local address 127.0.0.1:39526)
0926 21:23:26.575177 (+ 7290us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.575199 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.575390 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.576988 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.576991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.577009 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.578367 (+ 1358us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.578371 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.578388 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.578433 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.579058 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.579062 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.579067 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.579196 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.579247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7134,"thread_start_us":745,"threads_started":1}
I20260926 21:23:26.579901 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.575638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.14.142.3:41777)
0926 21:23:26.576437 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.576445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.577109 (+ 664us) server_negotiation.cc:407] Connection header received
0926 21:23:26.577265 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.577274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.577459 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.577623 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.578893 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.578903 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.578918 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.578939 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.578988 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.579008 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.579151 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.579397 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.579501 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":445,"threads_started":1}
I20260926 21:23:26.580355 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.568405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39526 (local address 127.14.142.1:46005)
0926 21:23:26.574387 (+ 5982us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.574412 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.575294 (+ 882us) server_negotiation.cc:407] Connection header received
0926 21:23:26.575574 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.575581 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.575778 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.575913 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.578726 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.578735 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.578749 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.578772 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.578837 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.578864 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.579000 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.579896 (+ 896us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.580013 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5736,"thread_start_us":5573,"threads_started":1}
I20260926 21:23:26.580935 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.576395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:41777 (local address 127.0.0.1:40902)
0926 21:23:26.576966 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.576994 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.577113 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.577621 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.577626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.577649 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.578658 (+ 1009us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.578661 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.578675 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.578715 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.579239 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.579243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.579247 (+ 4us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.580475 (+ 1228us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.580541 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":309,"threads_started":1}
I20260926 21:23:26.583931 19966 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb2718e24b24706aa8ba446f67e75d6 (DEFAULT_TABLE table=test-table [id=1bd13b64c30844cbaa8442777e0a8d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:26.584824 19705 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb2718e24b24706aa8ba446f67e75d6 (DEFAULT_TABLE table=test-table [id=1bd13b64c30844cbaa8442777e0a8d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:26.585634 19966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb2718e24b24706aa8ba446f67e75d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:26.584414 19836 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb2718e24b24706aa8ba446f67e75d6 (DEFAULT_TABLE table=test-table [id=1bd13b64c30844cbaa8442777e0a8d2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:26.586649 19836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb2718e24b24706aa8ba446f67e75d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:26.586683 19705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb2718e24b24706aa8ba446f67e75d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:26.606346 20049 tablet_bootstrap.cc:492] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Bootstrap starting.
I20260926 21:23:26.608009 20050 tablet_bootstrap.cc:492] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Bootstrap starting.
I20260926 21:23:26.609716 20051 tablet_bootstrap.cc:492] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Bootstrap starting.
I20260926 21:23:26.612833 20049 tablet_bootstrap.cc:654] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:26.613499 20051 tablet_bootstrap.cc:654] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:26.614293 20049 log.cc:824] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:26.614826 20051 log.cc:824] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:26.615262 19770 heartbeater.cc:503] Master 127.14.142.62:40761 was elected leader, sending a full tablet report...
I20260926 21:23:26.615617 20050 tablet_bootstrap.cc:654] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:26.617043 20050 log.cc:824] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:26.618005 20049 tablet_bootstrap.cc:492] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: No bootstrap required, opened a new log
I20260926 21:23:26.618429 20049 ts_tablet_manager.cc:1397] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260926 21:23:26.622781 20050 tablet_bootstrap.cc:492] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: No bootstrap required, opened a new log
I20260926 21:23:26.623078 20050 ts_tablet_manager.cc:1397] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260926 21:23:26.623312 20051 tablet_bootstrap.cc:492] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: No bootstrap required, opened a new log
I20260926 21:23:26.623620 20051 ts_tablet_manager.cc:1397] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260926 21:23:26.630306 20050 raft_consensus.cc:348] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.630741 20050 raft_consensus.cc:374] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:26.630882 20050 raft_consensus.cc:729] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7088e706b424c6ca7af646b7600f519, State: Initialized, Role: FOLLOWER
I20260926 21:23:26.630741 20049 raft_consensus.cc:348] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.631240 20049 raft_consensus.cc:374] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:26.631373 20049 raft_consensus.cc:729] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a438bdd401f4b6cb5d0c291ec300196, State: Initialized, Role: FOLLOWER
I20260926 21:23:26.631687 20050 consensus_queue.cc:260] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.632004 20049 consensus_queue.cc:260] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.636389 20049 ts_tablet_manager.cc:1428] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Time spent starting tablet: real 0.018s user 0.014s sys 0.002s
I20260926 21:23:26.636476 20050 ts_tablet_manager.cc:1428] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260926 21:23:26.636145 20051 raft_consensus.cc:348] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.636804 20051 raft_consensus.cc:374] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:26.636917 20051 raft_consensus.cc:729] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f6c67a304054d59a3adcb25051d25a0, State: Initialized, Role: FOLLOWER
I20260926 21:23:26.637324 20031 heartbeater.cc:503] Master 127.14.142.62:40761 was elected leader, sending a full tablet report...
I20260926 21:23:26.637732 20051 consensus_queue.cc:260] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.638906 19901 heartbeater.cc:503] Master 127.14.142.62:40761 was elected leader, sending a full tablet report...
I20260926 21:23:26.639631 20051 ts_tablet_manager.cc:1428] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260926 21:23:26.766441 20032 tablet.cc:2369] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:23:26.805964 19902 tablet.cc:2369] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:23:26.848711 19771 tablet.cc:2369] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:26.945474 20056 raft_consensus.cc:482] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:26.945875 20056 raft_consensus.cc:504] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.947623 20056 leader_election.cc:288] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f6c67a304054d59a3adcb25051d25a0 (127.14.142.2:38201), f7088e706b424c6ca7af646b7600f519 (127.14.142.1:46005)
I20260926 21:23:26.948894 20057 raft_consensus.cc:482] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:26.949344 20057 raft_consensus.cc:504] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.951184 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.948076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38201 (local address 127.14.142.3:43775)
0926 21:23:26.948305 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.948334 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.948457 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.949145 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.949153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.949176 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.950156 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.950162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.950185 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.950259 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.950721 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.950727 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.950732 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.950858 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.950913 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 21:23:26.951306 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.948091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:43775 (local address 127.14.142.2:38201)
0926 21:23:26.948313 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.948323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.948513 (+ 190us) server_negotiation.cc:407] Connection header received
0926 21:23:26.948761 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.948772 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.948892 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.949086 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.950424 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.950430 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.950433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.950480 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.950553 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.950574 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.950757 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.950884 (+ 127us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.950954 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 21:23:26.951859 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.948196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:43989 (local address 127.14.142.1:46005)
0926 21:23:26.948458 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.948464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.949436 (+ 972us) server_negotiation.cc:407] Connection header received
0926 21:23:26.949642 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.949648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.949706 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.949824 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.950943 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.950949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.950954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.950973 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.951008 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.951026 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.951120 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.951437 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.951502 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260926 21:23:26.952207 19856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb2718e24b24706aa8ba446f67e75d6" candidate_uuid: "2a438bdd401f4b6cb5d0c291ec300196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f6c67a304054d59a3adcb25051d25a0" is_pre_election: true
I20260926 21:23:26.952704 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.948419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46005 (local address 127.14.142.3:43989)
0926 21:23:26.949278 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.949297 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.949428 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.949842 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.949848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.949872 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.950701 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.950708 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.950726 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.950775 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.951222 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.951228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.951233 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.952040 (+ 807us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.952099 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":392,"threads_started":1}
I20260926 21:23:26.952821 19856 raft_consensus.cc:2454] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a438bdd401f4b6cb5d0c291ec300196 in term 0.
I20260926 21:23:26.953941 19919 leader_election.cc:302] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a438bdd401f4b6cb5d0c291ec300196, 2f6c67a304054d59a3adcb25051d25a0; no voters:
I20260926 21:23:26.954434 20056 raft_consensus.cc:2781] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:26.954599 20056 raft_consensus.cc:482] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:26.954710 20056 raft_consensus.cc:3037] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:26.954906 19725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb2718e24b24706aa8ba446f67e75d6" candidate_uuid: "2a438bdd401f4b6cb5d0c291ec300196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7088e706b424c6ca7af646b7600f519" is_pre_election: true
I20260926 21:23:26.955456 19725 raft_consensus.cc:2454] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a438bdd401f4b6cb5d0c291ec300196 in term 0.
I20260926 21:23:26.956400 20057 leader_election.cc:288] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a438bdd401f4b6cb5d0c291ec300196 (127.14.142.3:41777), f7088e706b424c6ca7af646b7600f519 (127.14.142.1:46005)
I20260926 21:23:26.957443 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.952267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:41777 (local address 127.14.142.2:39771)
0926 21:23:26.953786 (+ 1519us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.953811 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.954035 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.954949 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.954958 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.954995 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.955938 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.955944 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.955970 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.956037 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.956680 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.956689 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.956695 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.956890 (+ 195us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.956955 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":1124,"threads_started":1}
I20260926 21:23:26.957629 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.952349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:39771 (local address 127.14.142.3:41777)
0926 21:23:26.952574 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.952583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.953915 (+ 1332us) server_negotiation.cc:407] Connection header received
0926 21:23:26.954288 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.954297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.954378 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.954481 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.956279 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.956289 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.956297 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.956326 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.956380 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.956403 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.956601 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.957336 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.957395 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 21:23:26.958634 20056 raft_consensus.cc:504] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.958843 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb2718e24b24706aa8ba446f67e75d6" candidate_uuid: "2f6c67a304054d59a3adcb25051d25a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a438bdd401f4b6cb5d0c291ec300196" is_pre_election: true
I20260926 21:23:26.959434 19986 raft_consensus.cc:2379] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2f6c67a304054d59a3adcb25051d25a0 in current term 1: Already voted for candidate 2a438bdd401f4b6cb5d0c291ec300196 in this term.
I20260926 21:23:26.959530 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.954025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:45981 (local address 127.14.142.1:46005)
0926 21:23:26.954195 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:26.954202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:26.957170 (+ 2968us) server_negotiation.cc:407] Connection header received
0926 21:23:26.957305 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:26.957312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:26.957396 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:26.957507 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:26.958696 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:26.958704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:26.958709 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:26.958729 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:26.958775 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:26.958794 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:26.958937 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:26.959177 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 21:23:26.959238 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 21:23:26.959671 20056 leader_election.cc:288] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [CANDIDATE]: Term 1 election: Requested vote from peers 2f6c67a304054d59a3adcb25051d25a0 (127.14.142.2:38201), f7088e706b424c6ca7af646b7600f519 (127.14.142.1:46005)
I20260926 21:23:26.959736 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:26.953446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46005 (local address 127.14.142.2:45981)
0926 21:23:26.957042 (+ 3596us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:26.957066 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:26.957180 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:26.957667 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:26.957673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:26.957728 (+ 55us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:26.958458 (+ 730us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:23:26.958462 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:26.958480 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:23:26.958526 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:26.959001 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:26.959009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:26.959016 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 21:23:26.959182 (+ 166us) client_negotiation.cc:240] Negotiation successful
0926 21:23:26.959240 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3460,"thread_start_us":3214,"threads_started":1}
I20260926 21:23:26.960253 19725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb2718e24b24706aa8ba446f67e75d6" candidate_uuid: "2a438bdd401f4b6cb5d0c291ec300196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7088e706b424c6ca7af646b7600f519"
I20260926 21:23:26.960512 19725 raft_consensus.cc:3037] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:26.960546 19724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb2718e24b24706aa8ba446f67e75d6" candidate_uuid: "2f6c67a304054d59a3adcb25051d25a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7088e706b424c6ca7af646b7600f519" is_pre_election: true
I20260926 21:23:26.961419 19856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb2718e24b24706aa8ba446f67e75d6" candidate_uuid: "2a438bdd401f4b6cb5d0c291ec300196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f6c67a304054d59a3adcb25051d25a0"
I20260926 21:23:26.961651 19856 raft_consensus.cc:3037] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:26.961640 19791 leader_election.cc:302] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f6c67a304054d59a3adcb25051d25a0; no voters: 2a438bdd401f4b6cb5d0c291ec300196, f7088e706b424c6ca7af646b7600f519
I20260926 21:23:26.963922 19725 raft_consensus.cc:2454] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a438bdd401f4b6cb5d0c291ec300196 in term 1.
I20260926 21:23:26.964336 19921 leader_election.cc:302] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a438bdd401f4b6cb5d0c291ec300196, f7088e706b424c6ca7af646b7600f519; no voters:
I20260926 21:23:26.964679 20056 raft_consensus.cc:2781] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:26.964900 19856 raft_consensus.cc:2454] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a438bdd401f4b6cb5d0c291ec300196 in term 1.
I20260926 21:23:26.965138 20057 raft_consensus.cc:2726] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 21:23:26.965890 20056 raft_consensus.cc:686] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 1 LEADER]: Becoming Leader. State: Replica: 2a438bdd401f4b6cb5d0c291ec300196, State: Running, Role: LEADER
I20260926 21:23:26.966394 20056 consensus_queue.cc:237] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } }
I20260926 21:23:26.972882 19583 catalog_manager.cc:5495] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a438bdd401f4b6cb5d0c291ec300196 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "2a438bdd401f4b6cb5d0c291ec300196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a438bdd401f4b6cb5d0c291ec300196" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:27.037338 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:27.035040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.14.142.3:41777)
0926 21:23:27.035252 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:27.035258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:27.035280 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:23:27.035476 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:27.035480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:27.035741 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:27.035868 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:27.036840 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:27.036844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:27.036848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:27.036862 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:27.036895 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:27.036908 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:27.037025 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:27.037074 (+ 49us) server_negotiation.cc:299] Negotiation successful
0926 21:23:27.037120 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 21:23:27.045581 19725 raft_consensus.cc:1261] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 1 FOLLOWER]: Refusing update from remote peer 2a438bdd401f4b6cb5d0c291ec300196: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:27.045655 19856 raft_consensus.cc:1261] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 1 FOLLOWER]: Refusing update from remote peer 2a438bdd401f4b6cb5d0c291ec300196: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:27.046656 20056 consensus_queue.cc:1035] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7088e706b424c6ca7af646b7600f519" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:27.047063 20061 consensus_queue.cc:1035] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f6c67a304054d59a3adcb25051d25a0" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:27.074141 20064 mvcc.cc:205] Tried to move back new op lower bound from 7333715177648619520 to 7333715177339211776. Current Snapshot: MvccSnapshot[applied={T|T < 7333715177648619520 or (T in {7333715177648619520})}]
I20260926 21:23:27.074592 20067 mvcc.cc:205] Tried to move back new op lower bound from 7333715177648619520 to 7333715177339211776. Current Snapshot: MvccSnapshot[applied={T|T < 7333715177648619520}]
I20260926 21:23:27.086687 20065 mvcc.cc:205] Tried to move back new op lower bound from 7333715177648619520 to 7333715177339211776. Current Snapshot: MvccSnapshot[applied={T|T < 7333715177648619520}]
I20260926 21:23:27.089241 19584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:27.089522 19584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:27.094324 19584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9275cd05eb0c462ea7ed93098aa2bdbe: Sending DeleteTablet for 3 replicas of tablet 9eb2718e24b24706aa8ba446f67e75d6
I20260926 21:23:27.095700 19705 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb2718e24b24706aa8ba446f67e75d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:27 UTC) from {username='slave'} at 127.0.0.1:39526
I20260926 21:23:27.096143 19836 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb2718e24b24706aa8ba446f67e75d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:27 UTC) from {username='slave'} at 127.0.0.1:34672
I20260926 21:23:27.096807 20074 tablet_replica.cc:330] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: stopping tablet replica
I20260926 21:23:27.096753 19966 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb2718e24b24706aa8ba446f67e75d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:27 UTC) from {username='slave'} at 127.0.0.1:40902
I20260926 21:23:27.097259 20074 raft_consensus.cc:2229] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:27.097668 20075 tablet_replica.cc:330] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: stopping tablet replica
I20260926 21:23:27.097772 20074 raft_consensus.cc:2258] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:27.098156 20075 raft_consensus.cc:2229] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:27.098922 20075 raft_consensus.cc:2258] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:27.098979 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19643
I20260926 21:23:27.101635 20074 ts_tablet_manager.cc:1905] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:27.101647 20075 ts_tablet_manager.cc:1905] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:27.110247 20074 ts_tablet_manager.cc:1918] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:27.110445 20074 log.cc:1196] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/9eb2718e24b24706aa8ba446f67e75d6
I20260926 21:23:27.111019 20074 ts_tablet_manager.cc:1939] T 9eb2718e24b24706aa8ba446f67e75d6 P f7088e706b424c6ca7af646b7600f519: Deleting consensus metadata
I20260926 21:23:27.112946 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:27.101916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.14.142.1:46005)
0926 21:23:27.102107 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:27.102117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:27.109216 (+ 7099us) server_negotiation.cc:407] Connection header received
0926 21:23:27.109479 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:27.109485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:27.109575 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:27.109651 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:27.112020 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:27.112026 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:27.112031 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:27.112053 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:27.112101 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:27.112120 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:27.112233 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:27.112594 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 21:23:27.112660 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 21:23:27.113171 19570 catalog_manager.cc:4841] TS f7088e706b424c6ca7af646b7600f519 (127.14.142.1:46005): tablet 9eb2718e24b24706aa8ba446f67e75d6 (table test-table [id=1bd13b64c30844cbaa8442777e0a8d2a]) successfully deleted
I20260926 21:23:27.114262 19765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:27.116262 20075 ts_tablet_manager.cc:1918] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:27.116460 20075 log.cc:1196] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/9eb2718e24b24706aa8ba446f67e75d6
I20260926 21:23:27.117134 20075 ts_tablet_manager.cc:1939] T 9eb2718e24b24706aa8ba446f67e75d6 P 2a438bdd401f4b6cb5d0c291ec300196: Deleting consensus metadata
I20260926 21:23:27.119050 19568 catalog_manager.cc:4841] TS 2a438bdd401f4b6cb5d0c291ec300196 (127.14.142.3:41777): tablet 9eb2718e24b24706aa8ba446f67e75d6 (table test-table [id=1bd13b64c30844cbaa8442777e0a8d2a]) successfully deleted
I20260926 21:23:27.122258 20076 tablet_replica.cc:330] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: stopping tablet replica
I20260926 21:23:27.122750 20076 raft_consensus.cc:2229] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:27.123335 20076 raft_consensus.cc:2258] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:27.125686 20076 ts_tablet_manager.cc:1905] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:27.134023 20076 ts_tablet_manager.cc:1918] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:27.134256 20076 log.cc:1196] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/9eb2718e24b24706aa8ba446f67e75d6
I20260926 21:23:27.134824 20076 ts_tablet_manager.cc:1939] T 9eb2718e24b24706aa8ba446f67e75d6 P 2f6c67a304054d59a3adcb25051d25a0: Deleting consensus metadata
I20260926 21:23:27.136869 19568 catalog_manager.cc:4841] TS 2f6c67a304054d59a3adcb25051d25a0 (127.14.142.2:38201): tablet 9eb2718e24b24706aa8ba446f67e75d6 (table test-table [id=1bd13b64c30844cbaa8442777e0a8d2a]) successfully deleted
I20260926 21:23:27.218190 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19643
I20260926 21:23:27.244246 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19774
I20260926 21:23:27.247581 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:27.245305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34688 (local address 127.14.142.2:38201)
0926 21:23:27.245511 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:27.245515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:27.245535 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:23:27.245704 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:27.245715 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:27.245816 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:27.245912 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:27.246902 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:27.246906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:27.246909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:27.246923 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:27.246954 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:27.246965 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:27.247067 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:27.247200 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 21:23:27.247250 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260926 21:23:27.248258 19896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:27.345202 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19774
I20260926 21:23:27.364663 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19904
I20260926 21:23:27.368391 20047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:27.365839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.14.142.3:41777)
0926 21:23:27.366099 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:27.366105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:27.366129 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:23:27.366206 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:27.366217 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:27.366291 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:27.366424 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:27.367696 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:27.367702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:27.367707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:27.367727 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:27.367772 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:27.367789 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:27.367950 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:27.368010 (+ 60us) server_negotiation.cc:299] Negotiation successful
0926 21:23:27.368068 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 21:23:27.369397 20026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:27.495225 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19904
I20260926 21:23:27.515178 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 19553
I20260926 21:23:27.516517 19614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:27.630354 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 19553
2026-09-26T21:23:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3153 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 21:23:27.672696 14904 mini_oidc.cc:152] Starting JWKS server
I20260926 21:23:27.675573 14904 webserver.cc:460] Webserver started at https://127.0.0.1:45177/ using document root <none> and password file <none>
I20260926 21:23:27.675933 14904 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 21:23:27.676771 14904 webserver.cc:460] Webserver started at http://127.0.0.1:43483/ using document root <none> and password file <none>
2026-09-26T21:23:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:27Z Disabled control of system clock
I20260926 21:23:27.706125 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:43001
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:42761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:43001
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45177/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/testchainca.cert with env {}
W20260926 21:23:28.059952 20087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:28.060273 20087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:28.060366 20087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:28.067864 20087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:28.067989 20087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:28.068029 20087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:28.068084 20087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:28.068130 20087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 21:23:28.068193 20087 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 21:23:28.068233 20087 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45177/jwks.json
I20260926 21:23:28.080583 20087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:43001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:43001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45177/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:28.081950 20087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:28.083222 20087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:28.096060 20095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:28.096093 20093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:28.096144 20092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:28.102380 20087 server_base.cc:1034] running on GCE node
I20260926 21:23:28.103264 20087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:28.104905 20087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:28.106181 20087 hybrid_clock.cc:648] HybridClock initialized: now 1790457808106138 us; error 41 us; skew 500 ppm
I20260926 21:23:28.106585 20087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:28.109347 20087 webserver.cc:460] Webserver started at http://127.14.142.62:35603/ using document root <none> and password file <none>
I20260926 21:23:28.110121 20087 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:28.110349 20087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:28.110730 20087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:28.115005 20087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "f510a5f4deb54f2c85c484670c4b3794"
format_stamp: "Formatted at 2026-09-26 21:23:28 on dist-test-slave-4d5t"
server_key: "c19061c5f54dedc9a56db3a181c4bba7"
server_key_iv: "010e08fc38a5786dae4a1fa479bfe1c4"
server_key_version: "encryptionkey@0"
I20260926 21:23:28.115844 20087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "f510a5f4deb54f2c85c484670c4b3794"
format_stamp: "Formatted at 2026-09-26 21:23:28 on dist-test-slave-4d5t"
server_key: "c19061c5f54dedc9a56db3a181c4bba7"
server_key_iv: "010e08fc38a5786dae4a1fa479bfe1c4"
server_key_version: "encryptionkey@0"
I20260926 21:23:28.122629 20087 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 21:23:28.127413 20101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:28.129236 20087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:23:28.129460 20087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "f510a5f4deb54f2c85c484670c4b3794"
format_stamp: "Formatted at 2026-09-26 21:23:28 on dist-test-slave-4d5t"
server_key: "c19061c5f54dedc9a56db3a181c4bba7"
server_key_iv: "010e08fc38a5786dae4a1fa479bfe1c4"
server_key_version: "encryptionkey@0"
I20260926 21:23:28.129714 20087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:28.147632 20087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:28.148506 20087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:28.148850 20087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:28.168623 20087 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:43001
I20260926 21:23:28.168663 20152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:43001 every 8 connection(s)
I20260926 21:23:28.170040 20087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:28.175487 20153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:28.178830 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20087
I20260926 21:23:28.179040 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:28.179843 14904 external_mini_cluster.cc:1428] Setting key ebba4befdf67c7e38f47998babee918d
I20260926 21:23:28.190663 20153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794: Bootstrap starting.
I20260926 21:23:28.194705 20153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:28.195931 20153 log.cc:824] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:28.196959 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:28.182540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.14.142.62:43001)
0926 21:23:28.183992 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:28.184005 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:28.184053 (+ 48us) server_negotiation.cc:407] Connection header received
0926 21:23:28.185916 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:28.185946 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:28.186581 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:28.186971 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:28.187717 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:28.189228 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:28.191238 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:28.192322 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:28.192546 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:28.194016 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:28.194025 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:28.194041 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:28.194608 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:28.194708 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:28.194732 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:28.194899 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:28.195217 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 21:23:28.195693 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":566,"threads_started":1}
I20260926 21:23:28.198975 20153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794: No bootstrap required, opened a new log
I20260926 21:23:28.200802 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:23:28.207767 20153 raft_consensus.cc:348] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f510a5f4deb54f2c85c484670c4b3794" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43001 } }
I20260926 21:23:28.208206 20153 raft_consensus.cc:374] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:28.208316 20153 raft_consensus.cc:729] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f510a5f4deb54f2c85c484670c4b3794, State: Initialized, Role: FOLLOWER
I20260926 21:23:28.209020 20153 consensus_queue.cc:260] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f510a5f4deb54f2c85c484670c4b3794" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43001 } }
I20260926 21:23:28.209338 20153 raft_consensus.cc:388] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:28.209483 20153 raft_consensus.cc:482] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:28.209619 20153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:28.211786 20153 raft_consensus.cc:504] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f510a5f4deb54f2c85c484670c4b3794" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43001 } }
I20260926 21:23:28.212275 20153 leader_election.cc:302] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f510a5f4deb54f2c85c484670c4b3794; no voters:
I20260926 21:23:28.212899 20153 leader_election.cc:288] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:28.213085 20158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:28.214183 20158 raft_consensus.cc:686] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [term 1 LEADER]: Becoming Leader. State: Replica: f510a5f4deb54f2c85c484670c4b3794, State: Running, Role: LEADER
I20260926 21:23:28.214687 20158 consensus_queue.cc:237] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f510a5f4deb54f2c85c484670c4b3794" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43001 } }
I20260926 21:23:28.215162 20153 sys_catalog.cc:564] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:28.217859 20159 sys_catalog.cc:455] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f510a5f4deb54f2c85c484670c4b3794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f510a5f4deb54f2c85c484670c4b3794" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43001 } } }
I20260926 21:23:28.218303 20159 sys_catalog.cc:458] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:28.217837 20160 sys_catalog.cc:455] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f510a5f4deb54f2c85c484670c4b3794. Latest consensus state: current_term: 1 leader_uuid: "f510a5f4deb54f2c85c484670c4b3794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f510a5f4deb54f2c85c484670c4b3794" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43001 } } }
I20260926 21:23:28.219180 20160 sys_catalog.cc:458] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:28.220150 20165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:28.226373 20165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:28.237746 20165 catalog_manager.cc:1347] Generated new cluster ID: 5b33e2f7d6af45899317ad16102a0b4b
I20260926 21:23:28.237897 20165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:28.251384 20165 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:28.252825 20165 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:28.271095 20165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f510a5f4deb54f2c85c484670c4b3794: Generated new TSK 0
I20260926 21:23:28.272126 20165 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 21:23:28.333849 20183 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 21:23:28.334036 20183 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 21:23:28.337185 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:28.300036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.14.142.62:43001)
0926 21:23:28.300272 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:28.300278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:28.301264 (+ 986us) server_negotiation.cc:407] Connection header received
0926 21:23:28.303868 (+ 2604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:28.303875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:28.303977 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:28.304094 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0926 21:23:28.305068 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:28.306565 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:28.308726 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:28.309857 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:28.310017 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:28.310801 (+ 784us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 21:23:28.334595 (+ 23794us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 21:23:28.336889 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51106: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 21:23:28.337308 20156 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51106: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 21:23:28.337621 20182 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:28.299798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:43001 (local address 127.0.0.1:51106)
0926 21:23:28.301120 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:28.301142 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:28.301363 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:28.304364 (+ 3001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:28.304370 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:28.304384 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0926 21:23:28.304809 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:28.304822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:28.306717 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:28.306722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:28.308532 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:28.308546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:28.310065 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:28.310072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:28.310386 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:28.310498 (+ 112us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 21:23:28.335039 (+ 24541us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 21:23:28.335466 (+ 427us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.142.62:43001: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":1090,"thread_start_us":914,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.14.142.62:43001: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45177/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 21:23:28.339607 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20087
I20260926 21:23:28.340994 20148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:28.436671 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20087
2026-09-26T21:23:28Z chronyd exiting
I20260926 21:23:28.489439 14904 test_util.cc:175] -----------------------------------------------
I20260926 21:23:28.489816 14904 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790457761955429-14904-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (821 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 21:23:28.494032 14904 mini_oidc.cc:152] Starting JWKS server
I20260926 21:23:28.495911 14904 webserver.cc:460] Webserver started at https://127.0.0.1:46405/ using document root <none> and password file <none>
I20260926 21:23:28.496163 14904 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 21:23:28.496831 14904 webserver.cc:460] Webserver started at http://127.0.0.1:42285/ using document root <none> and password file <none>
2026-09-26T21:23:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:28Z Disabled control of system clock
I20260926 21:23:28.525885 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40113
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:40113
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46405/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/testchainca.cert with env {}
W20260926 21:23:28.881436 20190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:28.881739 20190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:28.881827 20190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:28.889218 20190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:28.889331 20190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:28.889370 20190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:28.889425 20190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:28.889470 20190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 21:23:28.889528 20190 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 21:23:28.889566 20190 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46405/jwks.json
I20260926 21:23:28.899770 20190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:40113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46405/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:28.901079 20190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:28.902287 20190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:28.914016 20198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:28.914130 20195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:28.914220 20196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:28.914865 20190 server_base.cc:1034] running on GCE node
I20260926 21:23:28.915748 20190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:28.917320 20190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:28.918576 20190 hybrid_clock.cc:648] HybridClock initialized: now 1790457808918496 us; error 57 us; skew 500 ppm
I20260926 21:23:28.918954 20190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:28.921676 20190 webserver.cc:460] Webserver started at http://127.14.142.62:40379/ using document root <none> and password file <none>
I20260926 21:23:28.922415 20190 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:28.922534 20190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:28.922878 20190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:28.926005 20190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "b9fa5fc0f47847dcb698e3c16cae2c04"
format_stamp: "Formatted at 2026-09-26 21:23:28 on dist-test-slave-4d5t"
server_key: "62d7438c3764bb6e2194de5b7865e549"
server_key_iv: "d7de67710a16926362f79a05305f639e"
server_key_version: "encryptionkey@0"
I20260926 21:23:28.926875 20190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "b9fa5fc0f47847dcb698e3c16cae2c04"
format_stamp: "Formatted at 2026-09-26 21:23:28 on dist-test-slave-4d5t"
server_key: "62d7438c3764bb6e2194de5b7865e549"
server_key_iv: "d7de67710a16926362f79a05305f639e"
server_key_version: "encryptionkey@0"
I20260926 21:23:28.932400 20190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:28.936681 20204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:28.938634 20190 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:28.938927 20190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "b9fa5fc0f47847dcb698e3c16cae2c04"
format_stamp: "Formatted at 2026-09-26 21:23:28 on dist-test-slave-4d5t"
server_key: "62d7438c3764bb6e2194de5b7865e549"
server_key_iv: "d7de67710a16926362f79a05305f639e"
server_key_version: "encryptionkey@0"
I20260926 21:23:28.939234 20190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:28.960222 20190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:28.961171 20190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:28.961483 20190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:28.982774 20190 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:40113
I20260926 21:23:28.982789 20255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:40113 every 8 connection(s)
I20260926 21:23:28.984203 20190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:28.989636 20256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:28.990481 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20190
I20260926 21:23:28.990900 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:28.991819 14904 external_mini_cluster.cc:1428] Setting key 48fd69a61d4e91440bbef471524fcf63
I20260926 21:23:29.006386 20256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04: Bootstrap starting.
I20260926 21:23:29.008366 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:28.994395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.14.142.62:40113)
0926 21:23:28.995825 (+ 1430us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:28.995838 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:28.995904 (+ 66us) server_negotiation.cc:407] Connection header received
0926 21:23:28.998195 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:28.998227 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:28.998831 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:28.999211 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:28.999842 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.001484 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.003647 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.004362 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.004513 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:29.005922 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:29.005930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:29.005946 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:29.006518 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:29.006661 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:29.006685 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:29.006786 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:29.007019 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 21:23:29.007416 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":863,"thread_start_us":551,"threads_started":1}
I20260926 21:23:29.010061 20256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:29.011265 20256 log.cc:824] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:29.012394 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:23:29.014628 20256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04: No bootstrap required, opened a new log
I20260926 21:23:29.023635 20256 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40113 } }
I20260926 21:23:29.024010 20256 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:29.024086 20256 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9fa5fc0f47847dcb698e3c16cae2c04, State: Initialized, Role: FOLLOWER
I20260926 21:23:29.024742 20256 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40113 } }
I20260926 21:23:29.025049 20256 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:29.025205 20256 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:29.025399 20256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:29.027637 20256 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40113 } }
I20260926 21:23:29.028093 20256 leader_election.cc:302] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9fa5fc0f47847dcb698e3c16cae2c04; no voters:
I20260926 21:23:29.028685 20256 leader_election.cc:288] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:29.028841 20261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:29.029626 20261 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [term 1 LEADER]: Becoming Leader. State: Replica: b9fa5fc0f47847dcb698e3c16cae2c04, State: Running, Role: LEADER
I20260926 21:23:29.030139 20261 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40113 } }
I20260926 21:23:29.030906 20256 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:29.033010 20263 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9fa5fc0f47847dcb698e3c16cae2c04. Latest consensus state: current_term: 1 leader_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40113 } } }
I20260926 21:23:29.033452 20263 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:29.033913 20262 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9fa5fc0f47847dcb698e3c16cae2c04" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40113 } } }
I20260926 21:23:29.034404 20262 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:29.037582 20270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:29.042332 20270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:29.054787 20270 catalog_manager.cc:1347] Generated new cluster ID: ede74749d2f04258a3392558b5bbac48
I20260926 21:23:29.054989 20270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:29.075223 20270 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:29.076699 20270 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:29.084448 20270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9fa5fc0f47847dcb698e3c16cae2c04: Generated new TSK 0
I20260926 21:23:29.085418 20270 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 21:23:29.123106 20286 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 21:23:29.123270 20286 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 21:23:29.124630 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:29.102623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.14.142.62:40113)
0926 21:23:29.102839 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:29.102847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:29.103339 (+ 492us) server_negotiation.cc:407] Connection header received
0926 21:23:29.103581 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:29.103588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:29.103691 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:29.103877 (+ 186us) server_negotiation.cc:226] Negotiated authn=JWT
0926 21:23:29.104371 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.105558 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.107502 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.108291 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.108438 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:29.109223 (+ 785us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 21:23:29.123820 (+ 14597us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46405/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 21:23:29.124261 (+ 441us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46405/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":77}
W20260926 21:23:29.124752 20259 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49800: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46405/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 21:23:29.125039 20285 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:29.102452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40113 (local address 127.0.0.1:49800)
0926 21:23:29.103219 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:29.103236 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:29.103382 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:29.103914 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:29.103920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:29.103930 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0926 21:23:29.104196 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:29.104203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.105843 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.105850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:29.107285 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:29.107304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.108619 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.108628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:29.108870 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:29.108922 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 21:23:29.124226 (+ 15304us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46405/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 21:23:29.124644 (+ 418us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.142.62:40113: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46405/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":444,"threads_started":1}
I20260926 21:23:29.127370 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20190
I20260926 21:23:29.128649 20251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:29.222159 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20190
2026-09-26T21:23:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (781 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 21:23:29.275959 14904 mini_oidc.cc:152] Starting JWKS server
I20260926 21:23:29.277976 14904 webserver.cc:460] Webserver started at https://127.0.0.1:45393/ using document root <none> and password file <none>
I20260926 21:23:29.278225 14904 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 21:23:29.278867 14904 webserver.cc:460] Webserver started at http://127.0.0.1:33151/ using document root <none> and password file <none>
2026-09-26T21:23:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:29Z Disabled control of system clock
I20260926 21:23:29.309136 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41833
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:37771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41833
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45393/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 21:23:29.647660 20293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:29.648032 20293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:29.648197 20293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:29.655610 20293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:29.655802 20293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:29.655893 20293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:29.656005 20293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:29.656095 20293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 21:23:29.656186 20293 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 21:23:29.656262 20293 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45393/jwks.json
I20260926 21:23:29.665763 20293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41833
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45393/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:29.667196 20293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:29.670064 20293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:29.683660 20301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:29.683745 20299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:29.683928 20298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:29.684813 20293 server_base.cc:1034] running on GCE node
I20260926 21:23:29.685667 20293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:29.687523 20293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:29.688855 20293 hybrid_clock.cc:648] HybridClock initialized: now 1790457809688783 us; error 56 us; skew 500 ppm
I20260926 21:23:29.689263 20293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:29.691870 20293 webserver.cc:460] Webserver started at http://127.14.142.62:39585/ using document root <none> and password file <none>
I20260926 21:23:29.692786 20293 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:29.692942 20293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:29.693308 20293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:29.696789 20293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "5156bb27c56541b39b5bde3080f02c30"
format_stamp: "Formatted at 2026-09-26 21:23:29 on dist-test-slave-4d5t"
server_key: "ff2e1b7e9d222caae2589e9e9b6b2175"
server_key_iv: "18a3bd30e5bbd0b41afa56a4ec3b8b62"
server_key_version: "encryptionkey@0"
I20260926 21:23:29.697710 20293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "5156bb27c56541b39b5bde3080f02c30"
format_stamp: "Formatted at 2026-09-26 21:23:29 on dist-test-slave-4d5t"
server_key: "ff2e1b7e9d222caae2589e9e9b6b2175"
server_key_iv: "18a3bd30e5bbd0b41afa56a4ec3b8b62"
server_key_version: "encryptionkey@0"
I20260926 21:23:29.703495 20293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 21:23:29.707284 20307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:29.709342 20293 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:23:29.709570 20293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "5156bb27c56541b39b5bde3080f02c30"
format_stamp: "Formatted at 2026-09-26 21:23:29 on dist-test-slave-4d5t"
server_key: "ff2e1b7e9d222caae2589e9e9b6b2175"
server_key_iv: "18a3bd30e5bbd0b41afa56a4ec3b8b62"
server_key_version: "encryptionkey@0"
I20260926 21:23:29.709851 20293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:29.735584 20293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:29.736517 20293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:29.736878 20293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:29.757887 20293 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:41833
I20260926 21:23:29.757947 20358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:41833 every 8 connection(s)
I20260926 21:23:29.759331 20293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:29.761507 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20293
I20260926 21:23:29.761785 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:29.762540 14904 external_mini_cluster.cc:1428] Setting key d5043154b7080680c872b4b4b1410b5f
I20260926 21:23:29.764247 20359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:29.778159 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:29.764804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.14.142.62:41833)
0926 21:23:29.765892 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:29.765903 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:29.765945 (+ 42us) server_negotiation.cc:407] Connection header received
0926 21:23:29.767540 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:29.767562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:29.768451 (+ 889us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:29.769164 (+ 713us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:29.769917 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.771498 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.773248 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.774009 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.774262 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:29.775674 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:29.775682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:29.775698 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:29.776319 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:29.776406 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:29.776429 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:29.776587 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:29.776832 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 21:23:29.777166 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":433,"threads_started":1}
I20260926 21:23:29.778167 20359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30: Bootstrap starting.
I20260926 21:23:29.781833 20359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:29.782178 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:23:29.782981 20359 log.cc:824] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:29.786115 20359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30: No bootstrap required, opened a new log
I20260926 21:23:29.794716 20359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5156bb27c56541b39b5bde3080f02c30" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41833 } }
I20260926 21:23:29.795164 20359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:29.795280 20359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5156bb27c56541b39b5bde3080f02c30, State: Initialized, Role: FOLLOWER
I20260926 21:23:29.795985 20359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5156bb27c56541b39b5bde3080f02c30" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41833 } }
I20260926 21:23:29.796281 20359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:29.796442 20359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:29.796656 20359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:29.799177 20359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5156bb27c56541b39b5bde3080f02c30" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41833 } }
I20260926 21:23:29.799741 20359 leader_election.cc:302] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5156bb27c56541b39b5bde3080f02c30; no voters:
I20260926 21:23:29.800380 20359 leader_election.cc:288] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:29.800589 20364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:29.801275 20364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [term 1 LEADER]: Becoming Leader. State: Replica: 5156bb27c56541b39b5bde3080f02c30, State: Running, Role: LEADER
I20260926 21:23:29.801976 20364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5156bb27c56541b39b5bde3080f02c30" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41833 } }
I20260926 21:23:29.803918 20359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:29.805188 20365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5156bb27c56541b39b5bde3080f02c30. Latest consensus state: current_term: 1 leader_uuid: "5156bb27c56541b39b5bde3080f02c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5156bb27c56541b39b5bde3080f02c30" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41833 } } }
I20260926 21:23:29.805215 20366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5156bb27c56541b39b5bde3080f02c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5156bb27c56541b39b5bde3080f02c30" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41833 } } }
I20260926 21:23:29.805706 20366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:29.805706 20365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:29.809492 20369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:29.815620 20369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:29.826628 20369 catalog_manager.cc:1347] Generated new cluster ID: ff7e6c30a5c747febf8ba03af24efffd
I20260926 21:23:29.826768 20369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:29.850448 20369 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:29.852180 20369 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:29.872027 20369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5156bb27c56541b39b5bde3080f02c30: Generated new TSK 0
I20260926 21:23:29.873281 20369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:23:29.898389 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:29.889398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40562 (local address 127.14.142.62:41833)
0926 21:23:29.889610 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:29.889617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:29.890016 (+ 399us) server_negotiation.cc:407] Connection header received
0926 21:23:29.890251 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:29.890258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:29.890370 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:29.890570 (+ 200us) server_negotiation.cc:226] Negotiated authn=JWT
0926 21:23:29.891014 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.892196 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.893718 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.894436 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.894615 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:29.895213 (+ 598us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 21:23:29.897707 (+ 2494us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 21:23:29.898059 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40562: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 21:23:29.898530 20362 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40562: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 21:23:29.900866 20388 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:29.889238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41833 (local address 127.0.0.1:40562)
0926 21:23:29.889909 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:29.889930 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:29.890068 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:29.890580 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:29.890583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:29.890592 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 21:23:29.890873 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:29.890881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.892373 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.892377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:29.893574 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:29.893585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:29.894696 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:29.894706 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:29.894866 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:29.894903 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 21:23:29.898013 (+ 3110us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 21:23:29.900391 (+ 2378us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.142.62:41833: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1775,"thread_start_us":387,"threads_started":1}
I20260926 21:23:29.901995 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20293
I20260926 21:23:29.903530 20354 generic_service.cc:196] Checking for leaks (request via RPC)
E20260926 21:23:29.911024 20389 webserver.cc:533] Webserver: sslize failed:
E20260926 21:23:29.911142 20389 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 21:23:30.000435 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20293
2026-09-26T21:23:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (781 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T21:23:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:30Z Disabled control of system clock
I20260926 21:23:30.085489 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44003
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44003
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/testchainca.cert with env {}
W20260926 21:23:30.428655 20394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:30.428982 20394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:30.429131 20394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:30.436651 20394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:30.436779 20394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:30.436820 20394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:30.436909 20394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:30.436976 20394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 21:23:30.437036 20394 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 21:23:30.437069 20394 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 21:23:30.447381 20394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:30.448865 20394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:30.450181 20394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:30.462950 20402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:30.463195 20399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:30.463403 20400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:30.464622 20394 server_base.cc:1034] running on GCE node
I20260926 21:23:30.465704 20394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:30.467347 20394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:30.468777 20394 hybrid_clock.cc:648] HybridClock initialized: now 1790457810468682 us; error 76 us; skew 500 ppm
I20260926 21:23:30.469185 20394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:23:30.471624 20394 webserver.cc:460] Webserver started at http://127.14.142.62:38511/ using document root <none> and password file <none>
I20260926 21:23:30.472355 20394 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:30.472504 20394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:30.472841 20394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:30.475823 20394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "6b1d656e69274cd38a1b258b0487bd9c"
format_stamp: "Formatted at 2026-09-26 21:23:30 on dist-test-slave-4d5t"
server_key: "1c5f90acafe0c8b7d36158a08d225ece"
server_key_iv: "4b22b54f07aee16821202f005b84cc3b"
server_key_version: "encryptionkey@0"
I20260926 21:23:30.476748 20394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "6b1d656e69274cd38a1b258b0487bd9c"
format_stamp: "Formatted at 2026-09-26 21:23:30 on dist-test-slave-4d5t"
server_key: "1c5f90acafe0c8b7d36158a08d225ece"
server_key_iv: "4b22b54f07aee16821202f005b84cc3b"
server_key_version: "encryptionkey@0"
I20260926 21:23:30.482148 20394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 21:23:30.485904 20408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:30.487466 20394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:30.487694 20394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "6b1d656e69274cd38a1b258b0487bd9c"
format_stamp: "Formatted at 2026-09-26 21:23:30 on dist-test-slave-4d5t"
server_key: "1c5f90acafe0c8b7d36158a08d225ece"
server_key_iv: "4b22b54f07aee16821202f005b84cc3b"
server_key_version: "encryptionkey@0"
I20260926 21:23:30.487960 20394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:30.506794 20394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:30.507712 20394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:30.508023 20394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:30.529246 20394 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:44003
I20260926 21:23:30.529296 20459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:44003 every 8 connection(s)
I20260926 21:23:30.530735 20394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:30.534876 20460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:30.539832 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20394
I20260926 21:23:30.540226 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:30.541046 14904 external_mini_cluster.cc:1428] Setting key 3675ba8685cae29df94b728aa70874e4
I20260926 21:23:30.549226 20460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c: Bootstrap starting.
I20260926 21:23:30.553553 20460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:30.555043 20460 log.cc:824] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:30.558248 20460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c: No bootstrap required, opened a new log
I20260926 21:23:30.567258 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:30.543378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.14.142.62:44003)
0926 21:23:30.544686 (+ 1308us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:30.544702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:30.544761 (+ 59us) server_negotiation.cc:407] Connection header received
0926 21:23:30.547534 (+ 2773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:30.547567 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:30.548244 (+ 677us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:30.548767 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:30.549634 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:30.551890 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:30.553396 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:30.554599 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:30.554831 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:30.563475 (+ 8644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:30.563494 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:30.563549 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:23:30.564641 (+ 1092us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:30.564766 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:23:30.564808 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:30.565121 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:30.565423 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 21:23:30.565828 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":418,"threads_started":1}
I20260926 21:23:30.567940 20460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1d656e69274cd38a1b258b0487bd9c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44003 } }
I20260926 21:23:30.568341 20460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:30.568470 20460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1d656e69274cd38a1b258b0487bd9c, State: Initialized, Role: FOLLOWER
I20260926 21:23:30.569222 20460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1d656e69274cd38a1b258b0487bd9c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44003 } }
I20260926 21:23:30.569590 20460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:30.569782 20460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:30.570026 20460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:30.571502 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:23:30.572968 20460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1d656e69274cd38a1b258b0487bd9c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44003 } }
I20260926 21:23:30.573557 20460 leader_election.cc:302] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b1d656e69274cd38a1b258b0487bd9c; no voters:
I20260926 21:23:30.574307 20460 leader_election.cc:288] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:30.574522 20465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:30.575587 20465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [term 1 LEADER]: Becoming Leader. State: Replica: 6b1d656e69274cd38a1b258b0487bd9c, State: Running, Role: LEADER
I20260926 21:23:30.576109 20465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1d656e69274cd38a1b258b0487bd9c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44003 } }
I20260926 21:23:30.576813 20460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:30.579049 20466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b1d656e69274cd38a1b258b0487bd9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1d656e69274cd38a1b258b0487bd9c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44003 } } }
I20260926 21:23:30.579475 20466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:30.579995 20467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b1d656e69274cd38a1b258b0487bd9c. Latest consensus state: current_term: 1 leader_uuid: "6b1d656e69274cd38a1b258b0487bd9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b1d656e69274cd38a1b258b0487bd9c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44003 } } }
I20260926 21:23:30.580266 20467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:30.586180 20474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:30.593268 20474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:30.603117 20474 catalog_manager.cc:1347] Generated new cluster ID: b3d00855ef52467fa1af703c926254de
I20260926 21:23:30.603343 20474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:30.612502 20474 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:30.614450 20474 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:30.627333 20474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b1d656e69274cd38a1b258b0487bd9c: Generated new TSK 0
I20260926 21:23:30.628762 20474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:23:30.680341 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:30.652911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.14.142.62:44003)
0926 21:23:30.653144 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:30.653151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:30.653567 (+ 416us) server_negotiation.cc:407] Connection header received
0926 21:23:30.653802 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:30.653808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:30.653917 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:30.654119 (+ 202us) server_negotiation.cc:226] Negotiated authn=JWT
0926 21:23:30.654590 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:30.655769 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:30.658149 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:30.658922 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:30.659099 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:30.659661 (+ 562us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 21:23:30.679478 (+ 19817us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 21:23:30.679971 (+ 493us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":85}
W20260926 21:23:30.680498 20463 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 21:23:30.680627 20489 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:30.652796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44003 (local address 127.0.0.1:44366)
0926 21:23:30.653453 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:30.653470 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:30.653604 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:30.654170 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:30.654174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:30.654183 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 21:23:30.654430 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:30.654437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:30.656063 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:30.656074 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:30.657956 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:30.657973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:30.659175 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:30.659180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:30.659319 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:30.659357 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 21:23:30.679851 (+ 20494us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 21:23:30.680175 (+ 324us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.14.142.62:44003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":360,"threads_started":1}
I20260926 21:23:30.682250 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20394
I20260926 21:23:30.683851 20455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:30.782747 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20394
2026-09-26T21:23:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (761 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (450 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (427 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:31 dist-test-slave-4d5t krb5kdc[20504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:31 dist-test-slave-4d5t krb5kdc[20504](info): set up 2 sockets
Sep 26 21:23:31 dist-test-slave-4d5t krb5kdc[20504](info): commencing operation
krb5kdc: starting...
W20260926 21:23:32.777652 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:32 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:33.008829 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:42565
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:42565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:33.351037 20517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:33.351362 20517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:33.351545 20517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:33.358996 20517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:33.359122 20517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:33.359172 20517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:33.359285 20517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:33.359371 20517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:33.369570 20517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:42565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:42565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:33.371865 20517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:33.373598 20517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:33.387176 20522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:33.387255 20523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:33.387274 20525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:33.389070 20517 server_base.cc:1034] running on GCE node
I20260926 21:23:33.390303 20517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:33.392249 20517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:33.393563 20517 hybrid_clock.cc:648] HybridClock initialized: now 1790457813393493 us; error 102 us; skew 500 ppm
I20260926 21:23:33.394013 20517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:33 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457813, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:33.397955 20517 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:33.399988 20517 webserver.cc:460] Webserver started at http://127.14.142.62:39065/ using document root <none> and password file <none>
I20260926 21:23:33.400841 20517 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:33.401003 20517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:33.401350 20517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:33.404640 20517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "e35915fd5c024a819f5067e3b1b88907"
format_stamp: "Formatted at 2026-09-26 21:23:33 on dist-test-slave-4d5t"
server_key: "89f819f2062ad2b32649b72070524b56"
server_key_iv: "7c9f65a960ea9a26a82a9483980899fa"
server_key_version: "encryptionkey@0"
I20260926 21:23:33.405494 20517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "e35915fd5c024a819f5067e3b1b88907"
format_stamp: "Formatted at 2026-09-26 21:23:33 on dist-test-slave-4d5t"
server_key: "89f819f2062ad2b32649b72070524b56"
server_key_iv: "7c9f65a960ea9a26a82a9483980899fa"
server_key_version: "encryptionkey@0"
I20260926 21:23:33.411108 20517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:33.415472 20532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:33.417148 20517 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:23:33.417341 20517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "e35915fd5c024a819f5067e3b1b88907"
format_stamp: "Formatted at 2026-09-26 21:23:33 on dist-test-slave-4d5t"
server_key: "89f819f2062ad2b32649b72070524b56"
server_key_iv: "7c9f65a960ea9a26a82a9483980899fa"
server_key_version: "encryptionkey@0"
I20260926 21:23:33.417562 20517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:33.455879 20517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:33.461745 20517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:33.462231 20517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:33.487263 20517 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:42565
I20260926 21:23:33.487291 20583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:42565 every 8 connection(s)
I20260926 21:23:33.488891 20517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:33.492785 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20517
I20260926 21:23:33.493076 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:33.493525 20584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:33.494009 14904 external_mini_cluster.cc:1428] Setting key a3d233d82c00f8990c639d0a5a78617c
I20260926 21:23:33.511096 20584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907: Bootstrap starting.
Sep 26 21:23:33 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:33.515138 20584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:33.516536 20584 log.cc:824] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:33.519816 20584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907: No bootstrap required, opened a new log
I20260926 21:23:33.522967 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:33.496705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.14.142.62:42565)
0926 21:23:33.497715 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:33.497727 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:33.497772 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:23:33.500653 (+ 2881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:33.500684 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:33.501291 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:33.501714 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:33.503692 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:33.505352 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:33.507635 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:33.508521 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:33.508761 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:33.514270 (+ 5509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:33.514279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:33.514296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:33.514346 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:33.518113 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:33.518966 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:33.518974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:33.518982 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:33.519165 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:33.519587 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:33.519599 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:33.519601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:33.520241 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:33.520619 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:33.521106 (+ 487us) server_negotiation.cc:299] Negotiation successful
0926 21:23:33.521648 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":485,"threads_started":1}
I20260926 21:23:33.529239 20584 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35915fd5c024a819f5067e3b1b88907" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42565 } }
I20260926 21:23:33.529773 20584 raft_consensus.cc:374] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:33.529946 20584 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35915fd5c024a819f5067e3b1b88907, State: Initialized, Role: FOLLOWER
I20260926 21:23:33.530726 20584 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35915fd5c024a819f5067e3b1b88907" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42565 } }
I20260926 21:23:33.531059 20584 raft_consensus.cc:388] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:33.531239 20584 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:33.531555 20584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:33.534754 20584 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35915fd5c024a819f5067e3b1b88907" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42565 } }
I20260926 21:23:33.535609 20584 leader_election.cc:302] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e35915fd5c024a819f5067e3b1b88907; no voters:
I20260926 21:23:33.536439 20584 leader_election.cc:288] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:33.536782 20589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:33.538105 20589 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [term 1 LEADER]: Becoming Leader. State: Replica: e35915fd5c024a819f5067e3b1b88907, State: Running, Role: LEADER
I20260926 21:23:33.538697 20589 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35915fd5c024a819f5067e3b1b88907" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42565 } }
I20260926 21:23:33.539090 20584 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:33.542472 20591 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35915fd5c024a819f5067e3b1b88907. Latest consensus state: current_term: 1 leader_uuid: "e35915fd5c024a819f5067e3b1b88907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35915fd5c024a819f5067e3b1b88907" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42565 } } }
I20260926 21:23:33.542943 20591 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:33.543463 20590 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e35915fd5c024a819f5067e3b1b88907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e35915fd5c024a819f5067e3b1b88907" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42565 } } }
I20260926 21:23:33.543860 20590 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:33.547369 20597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:33.553534 20597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:33.564253 20597 catalog_manager.cc:1347] Generated new cluster ID: 599cc731a8464c68a220087e292ddfc6
I20260926 21:23:33.564409 20597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:33.605552 20597 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:33.607620 20597 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:33.622255 20597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35915fd5c024a819f5067e3b1b88907: Generated new TSK 0
I20260926 21:23:33.623404 20597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:23:33.682276 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42565
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:34.049594 20610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:34.050037 20610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:34.050273 20610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:34.060499 20610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:34.060786 20610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:34.061044 20610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:34.074326 20610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:34.075729 20610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:34.077121 20610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:34.090969 20615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:34.091068 20618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:34.090987 20616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:34.092398 20610 server_base.cc:1034] running on GCE node
I20260926 21:23:34.093196 20610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:34.094213 20610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:34.095444 20610 hybrid_clock.cc:648] HybridClock initialized: now 1790457814095385 us; error 49 us; skew 500 ppm
I20260926 21:23:34.095765 20610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:34 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:34.099541 20610 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:23:34.101317 20610 webserver.cc:460] Webserver started at http://127.14.142.1:36873/ using document root <none> and password file <none>
I20260926 21:23:34.102104 20610 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:34.102268 20610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:34.102624 20610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:34.106173 20610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "cd58dd373c5f4e7b8187d6c8c481b956"
format_stamp: "Formatted at 2026-09-26 21:23:34 on dist-test-slave-4d5t"
server_key: "9edfef39a4cdbae78a07c84812a10c66"
server_key_iv: "15cb42048193c2089d76df1ddeffcded"
server_key_version: "encryptionkey@0"
I20260926 21:23:34.107074 20610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "cd58dd373c5f4e7b8187d6c8c481b956"
format_stamp: "Formatted at 2026-09-26 21:23:34 on dist-test-slave-4d5t"
server_key: "9edfef39a4cdbae78a07c84812a10c66"
server_key_iv: "15cb42048193c2089d76df1ddeffcded"
server_key_version: "encryptionkey@0"
I20260926 21:23:34.113032 20610 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:23:34.116779 20625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:34.118188 20610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:34.118384 20610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "cd58dd373c5f4e7b8187d6c8c481b956"
format_stamp: "Formatted at 2026-09-26 21:23:34 on dist-test-slave-4d5t"
server_key: "9edfef39a4cdbae78a07c84812a10c66"
server_key_iv: "15cb42048193c2089d76df1ddeffcded"
server_key_version: "encryptionkey@0"
I20260926 21:23:34.118675 20610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:34.137377 20610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:34.142706 20610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:34.143039 20610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:34.144116 20610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:34.145710 20610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:34.145823 20610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:34.145982 20610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:34.146061 20610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:34.183581 20610 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:35451
I20260926 21:23:34.183601 20737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:35451 every 8 connection(s)
I20260926 21:23:34.184985 20610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:34.190239 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20610
I20260926 21:23:34.190471 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:34.191097 14904 external_mini_cluster.cc:1428] Setting key b4f5c5138ee790cda02de262388b264c
Sep 26 21:23:34 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:34.209193 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:34.187688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:49193 (local address 127.14.142.62:42565)
0926 21:23:34.188157 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:34.188161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:34.189348 (+ 1187us) server_negotiation.cc:407] Connection header received
0926 21:23:34.191675 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:34.191682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:34.191798 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:34.191947 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:34.193785 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.194638 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.196601 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.197090 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.197268 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:34.202739 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:34.202745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:34.202748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:34.202801 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:34.205694 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:34.206706 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:34.206712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:34.206714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:34.206837 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:34.207314 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:34.207323 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:34.207325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:34.207613 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:34.207773 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:34.208548 (+ 775us) server_negotiation.cc:299] Negotiation successful
0926 21:23:34.208819 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":239,"threads_started":1}
I20260926 21:23:34.210229 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:34.188253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42565 (local address 127.14.142.1:49193)
0926 21:23:34.189167 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:34.189216 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:34.191359 (+ 2143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:34.192217 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:34.192227 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:34.192913 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:34.193582 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:34.193604 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.194873 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.194878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:34.196376 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:34.196393 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.197389 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.197400 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:34.197704 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:34.198576 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:34.198605 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:34.202377 (+ 3772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:34.205990 (+ 3613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:34.206002 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:34.206024 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:34.206544 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:34.206962 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:34.206967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:34.206970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:34.207183 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:34.207760 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:34.207768 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:34.208150 (+ 382us) client_negotiation.cc:769] Sending connection context
0926 21:23:34.208431 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 21:23:34.209044 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":416,"threads_started":1}
I20260926 21:23:34.212270 20738 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42565
I20260926 21:23:34.212667 20738 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:23:34.214044 20738 heartbeater.cc:511] Master 127.14.142.62:42565 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:23:34.216914 20549 ts_manager.cc:194] Registered new tserver with Master: cd58dd373c5f4e7b8187d6c8c481b956 (127.14.142.1:35451)
I20260926 21:23:34.219262 20549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:49193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:23:34.244634 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42565
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:34.576802 20744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:34.577082 20744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:34.577179 20744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:34.584399 20744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:34.584512 20744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:34.584702 20744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:34.594543 20744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:34.595795 20744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:34.597354 20744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:34.611640 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:34.611639 20750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:34.611640 20752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:34.612684 20744 server_base.cc:1034] running on GCE node
I20260926 21:23:34.613425 20744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:34.614523 20744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:34.615804 20744 hybrid_clock.cc:648] HybridClock initialized: now 1790457814615751 us; error 47 us; skew 500 ppm
I20260926 21:23:34.616204 20744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:34 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:34.619824 20744 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:23:34.621687 20744 webserver.cc:460] Webserver started at http://127.14.142.2:37273/ using document root <none> and password file <none>
I20260926 21:23:34.622740 20744 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:34.622903 20744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:34.623322 20744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:34.626915 20744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "8141f4ae66b8440cbad2da192421c9b1"
format_stamp: "Formatted at 2026-09-26 21:23:34 on dist-test-slave-4d5t"
server_key: "1853b5818286b6c5be8695130b316f46"
server_key_iv: "67b846c0a30c2219f31fba49fecfbafd"
server_key_version: "encryptionkey@0"
I20260926 21:23:34.627796 20744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "8141f4ae66b8440cbad2da192421c9b1"
format_stamp: "Formatted at 2026-09-26 21:23:34 on dist-test-slave-4d5t"
server_key: "1853b5818286b6c5be8695130b316f46"
server_key_iv: "67b846c0a30c2219f31fba49fecfbafd"
server_key_version: "encryptionkey@0"
I20260926 21:23:34.633459 20744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:34.637271 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:34.638887 20744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:23:34.639086 20744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "8141f4ae66b8440cbad2da192421c9b1"
format_stamp: "Formatted at 2026-09-26 21:23:34 on dist-test-slave-4d5t"
server_key: "1853b5818286b6c5be8695130b316f46"
server_key_iv: "67b846c0a30c2219f31fba49fecfbafd"
server_key_version: "encryptionkey@0"
I20260926 21:23:34.639310 20744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:34.668218 20744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:34.672982 20744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:34.673326 20744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:34.674417 20744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:34.675982 20744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:34.676069 20744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:34.676198 20744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:34.676259 20744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:34.715757 20744 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:46511
I20260926 21:23:34.715860 20871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:46511 every 8 connection(s)
I20260926 21:23:34.717108 20744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:34.727206 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20744
I20260926 21:23:34.727424 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:34.727964 14904 external_mini_cluster.cc:1428] Setting key 32799faba8ac9cef94acbf39211b456c
Sep 26 21:23:34 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457814, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:34.741112 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:34.719895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:36811 (local address 127.14.142.62:42565)
0926 21:23:34.720359 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:34.720366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:34.721386 (+ 1020us) server_negotiation.cc:407] Connection header received
0926 21:23:34.723488 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:34.723493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:34.723588 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:34.723695 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:34.726039 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.726894 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.729349 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.730049 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.730194 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:34.735111 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:34.735116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:34.735119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:34.735154 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:34.737760 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:34.738529 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:34.738534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:34.738536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:34.738684 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:34.739201 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:34.739206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:34.739209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:34.739487 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:34.739692 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:34.740459 (+ 767us) server_negotiation.cc:299] Negotiation successful
0926 21:23:34.740719 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":233,"threads_started":1}
I20260926 21:23:34.742153 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:34.720472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42565 (local address 127.14.142.2:36811)
0926 21:23:34.721192 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:34.721236 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:34.723095 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:34.724188 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:34.724200 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:34.725007 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:34.725777 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:34.725800 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.727135 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.727142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:34.729180 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:34.729201 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:34.730341 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:34.730351 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:34.730688 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:34.731585 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:34.731626 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:34.734831 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:34.737918 (+ 3087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:34.737928 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:34.737947 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:34.738389 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:34.738818 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:34.738823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:34.738827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:34.739077 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:34.739625 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:34.739633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:34.740039 (+ 406us) client_negotiation.cc:769] Sending connection context
0926 21:23:34.740358 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 21:23:34.740952 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":297,"threads_started":1}
I20260926 21:23:34.743952 20872 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42565
I20260926 21:23:34.744743 20872 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:34.745663 20872 heartbeater.cc:511] Master 127.14.142.62:42565 requested a full tablet report, sending...
I20260926 21:23:34.747649 20549 ts_manager.cc:194] Registered new tserver with Master: 8141f4ae66b8440cbad2da192421c9b1 (127.14.142.2:46511)
I20260926 21:23:34.748735 20549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:36811
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:23:34.784405 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42565
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:35.144922 20878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:35.145282 20878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:35.145468 20878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:35.154203 20878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:35.154424 20878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:35.154623 20878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:35.166659 20878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:35.168072 20878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:35.170272 20878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:35.186758 20884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:35.186785 20883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:35.189154 20886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:35.190575 20878 server_base.cc:1034] running on GCE node
I20260926 21:23:35.191586 20878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:35.192807 20878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:35.194151 20878 hybrid_clock.cc:648] HybridClock initialized: now 1790457815194072 us; error 47 us; skew 500 ppm
I20260926 21:23:35.194689 20878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.198652 20878 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:23:35.201087 20878 webserver.cc:460] Webserver started at http://127.14.142.3:42467/ using document root <none> and password file <none>
I20260926 21:23:35.201830 20878 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:35.201934 20878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:35.202203 20878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:35.205349 20878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "9f11f37854df4766a7a56bf425692c7a"
format_stamp: "Formatted at 2026-09-26 21:23:35 on dist-test-slave-4d5t"
server_key: "95042269afa8802be0d5b1069e35043d"
server_key_iv: "0129fdbd4d79aefcdeddc07e07a5dbde"
server_key_version: "encryptionkey@0"
I20260926 21:23:35.206243 20878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "9f11f37854df4766a7a56bf425692c7a"
format_stamp: "Formatted at 2026-09-26 21:23:35 on dist-test-slave-4d5t"
server_key: "95042269afa8802be0d5b1069e35043d"
server_key_iv: "0129fdbd4d79aefcdeddc07e07a5dbde"
server_key_version: "encryptionkey@0"
I20260926 21:23:35.211961 20878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 21:23:35.216415 20893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:35.218132 20878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:23:35.218348 20878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "9f11f37854df4766a7a56bf425692c7a"
format_stamp: "Formatted at 2026-09-26 21:23:35 on dist-test-slave-4d5t"
server_key: "95042269afa8802be0d5b1069e35043d"
server_key_iv: "0129fdbd4d79aefcdeddc07e07a5dbde"
server_key_version: "encryptionkey@0"
I20260926 21:23:35.218626 20878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:35.223170 20738 heartbeater.cc:503] Master 127.14.142.62:42565 was elected leader, sending a full tablet report...
I20260926 21:23:35.249836 20878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:35.254714 20878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:35.254976 20878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:35.255964 20878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:35.257537 20878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:35.257611 20878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:35.257695 20878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:35.257740 20878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:35.298218 20878 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:43043
I20260926 21:23:35.298372 21005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:43043 every 8 connection(s)
I20260926 21:23:35.299594 20878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:35.309844 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 20878
I20260926 21:23:35.310083 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:35.310590 14904 external_mini_cluster.cc:1428] Setting key bf2e08438582aa01caff9b2cb41f2e17
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:35.325081 21008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.302103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:50487 (local address 127.14.142.62:42565)
0926 21:23:35.302611 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.302615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.303378 (+ 763us) server_negotiation.cc:407] Connection header received
0926 21:23:35.305304 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.305310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.305403 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.305531 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.307035 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.308044 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.310595 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.311174 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.311334 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.316937 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.316943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.316947 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.316993 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.320699 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.321672 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.321677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.321679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.321819 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.322776 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.322781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.322783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.323109 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.323298 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.324463 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.324777 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":254,"threads_started":1}
I20260926 21:23:35.326252 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.302494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42565 (local address 127.14.142.3:50487)
0926 21:23:35.303228 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:35.303264 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:35.305032 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:35.305762 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:35.305771 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:35.306303 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:35.306861 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:35.306877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.308457 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.308463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:35.310402 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:35.310429 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.311528 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.311538 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:35.311964 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.312979 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:35.313023 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:35.316612 (+ 3589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:35.320881 (+ 4269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:35.320890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:35.320906 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:35.321482 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:35.322000 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:35.322005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:35.322008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:35.322533 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:35.323275 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:35.323285 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:35.323986 (+ 701us) client_negotiation.cc:769] Sending connection context
0926 21:23:35.324280 (+ 294us) client_negotiation.cc:240] Negotiation successful
0926 21:23:35.324883 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":277,"threads_started":1}
I20260926 21:23:35.328784 21006 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42565
I20260926 21:23:35.329154 21006 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:35.329917 21006 heartbeater.cc:511] Master 127.14.142.62:42565 requested a full tablet report, sending...
I20260926 21:23:35.331614 20548 ts_manager.cc:194] Registered new tserver with Master: 9f11f37854df4766a7a56bf425692c7a (127.14.142.3:43043)
I20260926 21:23:35.332726 20548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:50487
I20260926 21:23:35.341149 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/test-admin.keytab.
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.370486 14904 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 21:23:35.376434 21014 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:35.373513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.14.142.1:35451)
0926 21:23:35.374264 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.374271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.374299 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:23:35.374374 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.374383 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.374616 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.374798 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.375917 (+ 1119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52548: BlockingRecv error: recv got EOF from 127.0.0.1:52548 (error 108)
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":384,"threads_started":1}
W20260926 21:23:35.376575 14904 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc' of type FILE
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.378562 14904 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.380260 14904 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:35.398406 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.382184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.14.142.1:35451)
0926 21:23:35.382417 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.382424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.382555 (+ 131us) server_negotiation.cc:407] Connection header received
0926 21:23:35.382726 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.382730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.382793 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.382879 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.383943 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.385165 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.386969 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.387829 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.387957 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.391923 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.391937 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.391953 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.391992 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.395243 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.396001 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.396008 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.396010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.396138 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.396648 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.396652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.396654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.397052 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.397412 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.397812 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.398110 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260926 21:23:35.406379 21014 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:35.404207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.14.142.1:35451)
0926 21:23:35.404458 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.404465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.404735 (+ 270us) server_negotiation.cc:407] Connection header received
0926 21:23:35.404948 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.404954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.405060 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.405182 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.406054 (+ 872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52574: BlockingRecv error: recv got EOF from 127.0.0.1:52574 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260926 21:23:35.406653 14904 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc' of type FILE
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.408612 14904 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.410243 14904 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:35.425226 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.411700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.14.142.1:35451)
0926 21:23:35.411941 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.411948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.412197 (+ 249us) server_negotiation.cc:407] Connection header received
0926 21:23:35.412396 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.412401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.412479 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.412604 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.413609 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.414315 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.415530 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.415976 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.416121 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.419731 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.419736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.419739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.419774 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.422415 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.423206 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.423212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.423216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.423349 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.423787 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.423791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.423793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.424132 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.424409 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.424735 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.424946 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 21:23:35.430845 21014 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:35.428211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.14.142.1:35451)
0926 21:23:35.428611 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.428619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.429005 (+ 386us) server_negotiation.cc:407] Connection header received
0926 21:23:35.429268 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.429274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.429373 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.429489 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.430541 (+ 1052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52602: BlockingRecv error: recv got EOF from 127.0.0.1:52602 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
W20260926 21:23:35.431114 14904 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790457761955429-14904-0/krb5kdc/krb5cc' of type FILE
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.433144 14904 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:35.434551 14904 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:35.450911 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.435954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.14.142.1:35451)
0926 21:23:35.436209 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.436216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.436418 (+ 202us) server_negotiation.cc:407] Connection header received
0926 21:23:35.436693 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.436699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.436781 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.436904 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.437855 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.438581 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.439857 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.440283 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.440424 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.444289 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.444294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.444297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.444334 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.447529 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.448356 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.448361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.448364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.448632 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.449205 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.449210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.449213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.449643 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.449940 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.450332 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.450593 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:23:35.453083 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20610
I20260926 21:23:35.466400 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.454023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.14.142.1:35451)
0926 21:23:35.454205 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.454210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.454426 (+ 216us) server_negotiation.cc:407] Connection header received
0926 21:23:35.454537 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.454540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.454603 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.454693 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.455721 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.456440 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.457730 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.458234 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.458357 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.460789 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.460794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.460798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.460833 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.463730 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.464452 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.464456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.464459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.464701 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.465081 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.465085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.465087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.465377 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.465610 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.465911 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.466113 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 21:23:35.467080 20733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:35.554704 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20610
I20260926 21:23:35.571178 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20744
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:23:35.596592 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.572191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.14.142.2:46511)
0926 21:23:35.573170 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.573178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.573205 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:35.573270 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.573277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.573498 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.573650 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.574763 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.575929 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.577550 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.578145 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.578265 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.589882 (+ 11617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.589909 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.589926 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.589972 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.593032 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.593803 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.593811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.593814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.593969 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.594403 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.594407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.594409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.594799 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.595046 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.595593 (+ 547us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.595965 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":580,"threads_started":1}
I20260926 21:23:35.597975 20867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:35.682142 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20744
I20260926 21:23:35.698904 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20878
Sep 26 21:23:35 dist-test-slave-4d5t krb5kdc[20504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:23:35.717023 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:35.700078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.14.142.3:43043)
0926 21:23:35.700801 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:35.700808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:35.700839 (+ 31us) server_negotiation.cc:407] Connection header received
0926 21:23:35.700900 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:35.700907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:35.701123 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:35.701256 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:35.702307 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.703394 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.704861 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:35.705815 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:35.705926 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:35.710494 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:35.710506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:35.710521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:35.710561 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:35.713765 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.714914 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.714923 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.714925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.715093 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:35.715526 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:35.715530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:35.715532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:35.715927 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:35.716171 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:35.716414 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 21:23:35.716726 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":349,"threads_started":1}
I20260926 21:23:35.718289 21001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:35.803699 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20878
I20260926 21:23:35.821439 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 20517
I20260926 21:23:35.823153 20579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:35.934103 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 20517
2026-09-26T21:23:35Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4280 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:36 dist-test-slave-4d5t krb5kdc[21034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:36 dist-test-slave-4d5t krb5kdc[21034](info): set up 2 sockets
Sep 26 21:23:36 dist-test-slave-4d5t krb5kdc[21034](info): commencing operation
krb5kdc: starting...
W20260926 21:23:37.059845 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:37 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab.
Entry for principal oryx/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab.
I20260926 21:23:37.296496 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:42663
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab
--principal=oryx/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:42663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:37.634792 21047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:37.635169 21047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:37.635345 21047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:37.642853 21047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:37.643023 21047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:37.643137 21047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:37.643244 21047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:37.643335 21047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:37.653942 21047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:42663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab
--principal=oryx/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:42663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:37.655380 21047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:37.656702 21047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:37.670169 21052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:37.670095 21053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:37.670127 21055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:37.671171 21047 server_base.cc:1034] running on GCE node
I20260926 21:23:37.672143 21047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:37.673794 21047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:37.675314 21047 hybrid_clock.cc:648] HybridClock initialized: now 1790457817675026 us; error 39 us; skew 500 ppm
I20260926 21:23:37.676010 21047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:37 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:37.680182 21047 init.cc:377] Logged in from keytab as oryx/127.14.142.62@KRBTEST.COM (short username oryx)
I20260926 21:23:37.682132 21047 webserver.cc:460] Webserver started at http://127.14.142.62:33485/ using document root <none> and password file <none>
I20260926 21:23:37.682932 21047 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:37.683269 21047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:37.683676 21047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:37.686844 21047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "b1e509ff7dc4430eb1417ebf8ed92b46"
format_stamp: "Formatted at 2026-09-26 21:23:37 on dist-test-slave-4d5t"
server_key: "813d9202e721e0fb83701fb7da4315d6"
server_key_iv: "79bf88b997ac8c2bbe4a9987264d1fcb"
server_key_version: "encryptionkey@0"
I20260926 21:23:37.687669 21047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "b1e509ff7dc4430eb1417ebf8ed92b46"
format_stamp: "Formatted at 2026-09-26 21:23:37 on dist-test-slave-4d5t"
server_key: "813d9202e721e0fb83701fb7da4315d6"
server_key_iv: "79bf88b997ac8c2bbe4a9987264d1fcb"
server_key_version: "encryptionkey@0"
I20260926 21:23:37.693254 21047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 21:23:37.697074 21062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:37.698799 21047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:23:37.699038 21047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "b1e509ff7dc4430eb1417ebf8ed92b46"
format_stamp: "Formatted at 2026-09-26 21:23:37 on dist-test-slave-4d5t"
server_key: "813d9202e721e0fb83701fb7da4315d6"
server_key_iv: "79bf88b997ac8c2bbe4a9987264d1fcb"
server_key_version: "encryptionkey@0"
I20260926 21:23:37.699281 21047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:37.748319 21047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:37.754227 21047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:37.754576 21047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:37.775707 21047 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:42663
I20260926 21:23:37.775729 21113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:42663 every 8 connection(s)
I20260926 21:23:37.777179 21047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:37.780483 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21047
I20260926 21:23:37.780768 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:37.781255 21114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:37.781644 14904 external_mini_cluster.cc:1428] Setting key ab17b828cd0bcad1a95a359df0693ffc
I20260926 21:23:37.795554 21114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Bootstrap starting.
Sep 26 21:23:37 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:37.799552 21114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:37.801038 21114 log.cc:824] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:37.804534 21114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: No bootstrap required, opened a new log
I20260926 21:23:37.808457 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:37.783946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.14.142.62:42663)
0926 21:23:37.784852 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:37.784875 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:37.784921 (+ 46us) server_negotiation.cc:407] Connection header received
0926 21:23:37.787390 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:37.787415 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:37.787852 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:37.788224 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:37.790110 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:37.791475 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:37.793440 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:37.794468 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:37.794687 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:37.800091 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:37.800103 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:37.800130 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:37.800198 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:37.803954 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:37.804991 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:37.805002 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:37.805008 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:37.805188 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:37.805597 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:37.805601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:37.805603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:37.806145 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:37.806522 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:37.806943 (+ 421us) server_negotiation.cc:299] Negotiation successful
0926 21:23:37.807394 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":426,"threads_started":1}
I20260926 21:23:37.813776 21114 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } }
I20260926 21:23:37.814249 21114 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:37.814363 21114 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1e509ff7dc4430eb1417ebf8ed92b46, State: Initialized, Role: FOLLOWER
I20260926 21:23:37.814994 21114 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } }
I20260926 21:23:37.815266 21114 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:37.815407 21114 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:37.815589 21114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:37.818141 21114 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } }
I20260926 21:23:37.818681 21114 leader_election.cc:302] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1e509ff7dc4430eb1417ebf8ed92b46; no voters:
I20260926 21:23:37.819345 21114 leader_election.cc:288] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:37.819517 21119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:37.820204 21119 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 LEADER]: Becoming Leader. State: Replica: b1e509ff7dc4430eb1417ebf8ed92b46, State: Running, Role: LEADER
I20260926 21:23:37.820833 21119 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } }
I20260926 21:23:37.821625 21114 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:37.823685 21120 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } }
I20260926 21:23:37.824096 21120 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:37.824416 21121 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e509ff7dc4430eb1417ebf8ed92b46. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } }
I20260926 21:23:37.824853 21121 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:37.827809 21124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:37.835654 21124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:37.847128 21124 catalog_manager.cc:1347] Generated new cluster ID: 1ccf6a148e6e4d4db595e57b7ebb5b99
I20260926 21:23:37.847287 21124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:37.872821 21124 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:37.874248 21124 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:37.887738 21124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Generated new TSK 0
I20260926 21:23:37.888816 21124 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab.
Entry for principal oryx/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab.
I20260926 21:23:37.952068 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42663
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:38.327788 21140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:38.328078 21140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:38.328171 21140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:38.335630 21140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:38.335745 21140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:38.335860 21140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:38.345597 21140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:38.346899 21140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:38.348214 21140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:38.361609 21148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:38.362134 21145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:38.362231 21146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:38.362610 21140 server_base.cc:1034] running on GCE node
I20260926 21:23:38.363672 21140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:38.365396 21140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:38.366667 21140 hybrid_clock.cc:648] HybridClock initialized: now 1790457818366543 us; error 141 us; skew 500 ppm
I20260926 21:23:38.367250 21140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:38 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457818, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:38.372150 21140 init.cc:377] Logged in from keytab as oryx/127.14.142.1@KRBTEST.COM (short username oryx)
I20260926 21:23:38.376361 21140 webserver.cc:460] Webserver started at http://127.14.142.1:40881/ using document root <none> and password file <none>
I20260926 21:23:38.377333 21140 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:38.377485 21140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:38.377875 21140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:38.381614 21140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "176d2d568df44be4b8bfe5ee05793052"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "7317870f27fd4357a55fb42ff7a3b9c4"
server_key_iv: "5ae79a71ea1cb5e7a996fdff9c421d61"
server_key_version: "encryptionkey@0"
I20260926 21:23:38.382762 21140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "176d2d568df44be4b8bfe5ee05793052"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "7317870f27fd4357a55fb42ff7a3b9c4"
server_key_iv: "5ae79a71ea1cb5e7a996fdff9c421d61"
server_key_version: "encryptionkey@0"
I20260926 21:23:38.390106 21140 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260926 21:23:38.394936 21155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:38.397073 21140 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260926 21:23:38.397281 21140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "176d2d568df44be4b8bfe5ee05793052"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "7317870f27fd4357a55fb42ff7a3b9c4"
server_key_iv: "5ae79a71ea1cb5e7a996fdff9c421d61"
server_key_version: "encryptionkey@0"
I20260926 21:23:38.397580 21140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:38.423743 21140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:38.428105 21140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:38.428411 21140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:38.429466 21140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:38.430944 21140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:38.431051 21140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:38.431203 21140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:38.431277 21140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:38.467108 21140 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:36175
I20260926 21:23:38.467196 21267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:36175 every 8 connection(s)
I20260926 21:23:38.468477 21140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:38.478830 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21140
I20260926 21:23:38.479099 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:38.479741 14904 external_mini_cluster.cc:1428] Setting key 593dad250dd7697d8f759e05dd8993ee
Sep 26 21:23:38 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457818, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:38.489881 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:38.471174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:39075 (local address 127.14.142.62:42663)
0926 21:23:38.471609 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:38.471614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:38.472667 (+ 1053us) server_negotiation.cc:407] Connection header received
0926 21:23:38.474568 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:38.474574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:38.474672 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:38.474795 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:38.476361 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:38.477154 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:38.478800 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:38.479362 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:38.479497 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:38.483936 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:38.483947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:38.483950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:38.483985 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:38.486635 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:38.487439 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:38.487448 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:38.487452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:38.487578 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:38.488016 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:38.488023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:38.488026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:38.488294 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:38.488455 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:38.489265 (+ 810us) server_negotiation.cc:299] Negotiation successful
0926 21:23:38.489497 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":206,"threads_started":1}
I20260926 21:23:38.491073 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:38.471731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:39075)
0926 21:23:38.472453 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:38.472497 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:38.474271 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:38.475045 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:38.475054 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:38.475597 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:38.476173 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:38.476190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:38.477317 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:38.477323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:38.478642 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:38.478654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:38.479600 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:38.479607 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:38.479877 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:38.480754 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:38.480783 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:38.483672 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:38.486811 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:38.486821 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:38.486841 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:38.487297 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:38.487696 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:38.487702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:38.487705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:38.487901 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:38.488427 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:38.488435 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:38.488881 (+ 446us) client_negotiation.cc:769] Sending connection context
0926 21:23:38.489167 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 21:23:38.489750 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":298,"threads_started":1}
I20260926 21:23:38.493124 21268 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42663
I20260926 21:23:38.493466 21268 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:38.494236 21268 heartbeater.cc:511] Master 127.14.142.62:42663 requested a full tablet report, sending...
I20260926 21:23:38.497007 21079 ts_manager.cc:194] Registered new tserver with Master: 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175)
I20260926 21:23:38.498507 21079 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:39075
WARNING: no policy specified for oryx/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab.
Entry for principal oryx/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab.
I20260926 21:23:38.533344 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42663
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:38.877596 21274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:38.877907 21274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:38.878002 21274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:38.886476 21274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:38.886611 21274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:38.886729 21274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:38.898226 21274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:38.900141 21274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:38.901728 21274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:38.916160 21279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:38.916543 21280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:38.916610 21282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:38.917548 21274 server_base.cc:1034] running on GCE node
I20260926 21:23:38.918290 21274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:38.919381 21274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:38.920673 21274 hybrid_clock.cc:648] HybridClock initialized: now 1790457818920609 us; error 56 us; skew 500 ppm
I20260926 21:23:38.921028 21274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:38 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457818, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:38.925017 21274 init.cc:377] Logged in from keytab as oryx/127.14.142.2@KRBTEST.COM (short username oryx)
I20260926 21:23:38.926620 21274 webserver.cc:460] Webserver started at http://127.14.142.2:42713/ using document root <none> and password file <none>
I20260926 21:23:38.927338 21274 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:38.927497 21274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:38.927807 21274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:38.931113 21274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "d16ca4cc2861428faad7658e6f5f7445"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "4d433922a55bbd9d2a0b3d2d374e242d"
server_key_iv: "929b24540aa1641378ae3b735b686e64"
server_key_version: "encryptionkey@0"
I20260926 21:23:38.931929 21274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "d16ca4cc2861428faad7658e6f5f7445"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "4d433922a55bbd9d2a0b3d2d374e242d"
server_key_iv: "929b24540aa1641378ae3b735b686e64"
server_key_version: "encryptionkey@0"
I20260926 21:23:38.937817 21274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 21:23:38.941498 21289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:38.942966 21274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:38.943178 21274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "d16ca4cc2861428faad7658e6f5f7445"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "4d433922a55bbd9d2a0b3d2d374e242d"
server_key_iv: "929b24540aa1641378ae3b735b686e64"
server_key_version: "encryptionkey@0"
I20260926 21:23:38.943413 21274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:38.977355 21274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:38.982180 21274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:38.982518 21274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:38.983520 21274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:38.984982 21274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:38.985086 21274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:38.985239 21274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:38.985312 21274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:39.022117 21274 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:41363
I20260926 21:23:39.022156 21401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:41363 every 8 connection(s)
I20260926 21:23:39.023650 21274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:39.027513 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21274
I20260926 21:23:39.027737 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:39.028275 14904 external_mini_cluster.cc:1428] Setting key 676913088f7197b7002117071d640e07
Sep 26 21:23:39 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457818, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:39.048002 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:39.026676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:58885 (local address 127.14.142.62:42663)
0926 21:23:39.027481 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:39.027492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:39.028230 (+ 738us) server_negotiation.cc:407] Connection header received
0926 21:23:39.030739 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:39.030745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:39.030835 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:39.030958 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:39.032474 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.033270 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.035101 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.035626 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.035780 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:39.039821 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:39.039832 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:39.039841 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:39.039893 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:39.043860 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:39.044667 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:39.044676 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:39.044683 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:39.044866 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:39.045335 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:39.045343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:39.045350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:39.045766 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:39.045935 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:39.047114 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0926 21:23:39.047440 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":467,"threads_started":1}
I20260926 21:23:39.049649 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:39.026887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:58885)
0926 21:23:39.028039 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:39.028087 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:39.030462 (+ 2375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:39.031207 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:39.031215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:39.031731 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:39.032306 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:39.032322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.033424 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.033429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:39.034899 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:39.034913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.035894 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.035902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:39.036157 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:39.036989 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:39.037015 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:39.039544 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:39.044026 (+ 4482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:39.044033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:39.044048 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:39.044483 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:39.044999 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:39.045003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:39.045005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:39.045188 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:39.046012 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:39.046022 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:39.046565 (+ 543us) client_negotiation.cc:769] Sending connection context
0926 21:23:39.046981 (+ 416us) client_negotiation.cc:240] Negotiation successful
0926 21:23:39.047696 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":484,"threads_started":1}
I20260926 21:23:39.052724 21402 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42663
I20260926 21:23:39.053318 21402 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:23:39.054556 21402 heartbeater.cc:511] Master 127.14.142.62:42663 requested a full tablet report, sending...
I20260926 21:23:39.056515 21079 ts_manager.cc:194] Registered new tserver with Master: d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.3@KRBTEST.COM" created.
I20260926 21:23:39.057729 21079 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:58885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab.
Entry for principal oryx/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab.
I20260926 21:23:39.084233 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42663
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:39.422912 21408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:39.423370 21408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:39.423486 21408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:39.431870 21408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:39.432024 21408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:39.432155 21408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:39.442543 21408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:42663
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:39.443995 21408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:39.445489 21408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:39.459136 21414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:39.459244 21416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:39.459226 21413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:39.465894 21408 server_base.cc:1034] running on GCE node
I20260926 21:23:39.466706 21408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:39.468819 21408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:39.470084 21408 hybrid_clock.cc:648] HybridClock initialized: now 1790457819470034 us; error 43 us; skew 500 ppm
I20260926 21:23:39.470479 21408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:39 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457819, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:39.474509 21408 init.cc:377] Logged in from keytab as oryx/127.14.142.3@KRBTEST.COM (short username oryx)
I20260926 21:23:39.476370 21408 webserver.cc:460] Webserver started at http://127.14.142.3:44331/ using document root <none> and password file <none>
I20260926 21:23:39.477200 21408 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:39.477355 21408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:39.477672 21408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:39.480827 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "25bb5e784e894167804079ec5800aa57"
format_stamp: "Formatted at 2026-09-26 21:23:39 on dist-test-slave-4d5t"
server_key: "c04697199dde06911b87fb3832267795"
server_key_iv: "7ad3a400a9a00502ffc7ce29b113e332"
server_key_version: "encryptionkey@0"
I20260926 21:23:39.481675 21408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "25bb5e784e894167804079ec5800aa57"
format_stamp: "Formatted at 2026-09-26 21:23:39 on dist-test-slave-4d5t"
server_key: "c04697199dde06911b87fb3832267795"
server_key_iv: "7ad3a400a9a00502ffc7ce29b113e332"
server_key_version: "encryptionkey@0"
I20260926 21:23:39.487654 21408 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:23:39.492048 21423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:39.493748 21408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:23:39.493984 21408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "25bb5e784e894167804079ec5800aa57"
format_stamp: "Formatted at 2026-09-26 21:23:39 on dist-test-slave-4d5t"
server_key: "c04697199dde06911b87fb3832267795"
server_key_iv: "7ad3a400a9a00502ffc7ce29b113e332"
server_key_version: "encryptionkey@0"
I20260926 21:23:39.494228 21408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:39.501904 21268 heartbeater.cc:503] Master 127.14.142.62:42663 was elected leader, sending a full tablet report...
I20260926 21:23:39.512108 21408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:39.517014 21408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:39.517334 21408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:39.518383 21408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:39.519874 21408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:39.520011 21408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:39.520167 21408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:39.520251 21408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:39.559154 21408 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:37659
I20260926 21:23:39.559204 21535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:37659 every 8 connection(s)
I20260926 21:23:39.560504 21408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:39.567615 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21408
I20260926 21:23:39.567833 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:39.568382 14904 external_mini_cluster.cc:1428] Setting key ea6cbd33b7f42cbb31add112180c5dbf
Sep 26 21:23:39 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457819, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:39.584877 21538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:39.563669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:58877 (local address 127.14.142.62:42663)
0926 21:23:39.564280 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:39.564285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:39.565369 (+ 1084us) server_negotiation.cc:407] Connection header received
0926 21:23:39.567423 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:39.567428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:39.567512 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:39.567614 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:39.569751 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.570846 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.572981 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.573646 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.573786 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:39.578041 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:39.578048 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:39.578052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:39.578091 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:39.580873 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:39.581715 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:39.581719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:39.581721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:39.581843 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:39.582358 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:39.582362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:39.582365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:39.582685 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:39.582854 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:39.584084 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0926 21:23:39.584461 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":312,"threads_started":1}
I20260926 21:23:39.585583 21539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:39.564376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:58877)
0926 21:23:39.565195 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:39.565233 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:39.566943 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:39.567917 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:39.567939 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:39.568730 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:39.569560 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:39.569583 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.571063 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.571068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:39.572733 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:39.572749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:39.573923 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:39.573928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:39.574174 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:39.574914 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:39.574946 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:39.577739 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:39.581038 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:39.581046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:39.581062 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:39.581588 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:39.581951 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:39.581955 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:39.581959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:39.582225 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:39.582847 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:39.582853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:39.583376 (+ 523us) client_negotiation.cc:769] Sending connection context
0926 21:23:39.583751 (+ 375us) client_negotiation.cc:240] Negotiation successful
0926 21:23:39.584231 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":363,"threads_started":1}
I20260926 21:23:39.587152 21536 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42663
I20260926 21:23:39.587548 21536 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:39.588367 21536 heartbeater.cc:511] Master 127.14.142.62:42663 requested a full tablet report, sending...
I20260926 21:23:39.590054 21079 ts_manager.cc:194] Registered new tserver with Master: 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:39.590920 21079 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:58877
I20260926 21:23:39.597364 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.142.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.61.keytab.
Entry for principal oryx/127.14.142.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.61.keytab.
I20260926 21:23:39.652873 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:33303
--webserver_interface=127.14.142.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.61.keytab
--principal=oryx/127.14.142.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:42663,127.14.142.61:33303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:40.006438 21542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:40.006753 21542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:40.006903 21542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:40.014537 21542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:40.014655 21542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:40.014701 21542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:40.014792 21542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:40.014865 21542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:40.028501 21542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:42663,127.14.142.61:33303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.61.keytab
--principal=oryx/127.14.142.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:33303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.142.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
W20260926 21:23:40.029089 21542 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.14.142.62:42663,127.14.142.61:33303'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 21:23:40.030395 21542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:40.031751 21542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:40.044608 21548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:40.044641 21550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:40.044711 21547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:40.046126 21542 server_base.cc:1034] running on GCE node
I20260926 21:23:40.047160 21542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:40.048974 21542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:40.050282 21542 hybrid_clock.cc:648] HybridClock initialized: now 1790457820050179 us; error 44 us; skew 500 ppm
I20260926 21:23:40.050674 21542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:40.055742 21542 init.cc:377] Logged in from keytab as oryx/127.14.142.61@KRBTEST.COM (short username oryx)
I20260926 21:23:40.058139 21542 webserver.cc:460] Webserver started at http://127.14.142.61:38801/ using document root <none> and password file <none>
I20260926 21:23:40.059026 21542 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:40.059128 21542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:40.059506 21542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:40.060396 21402 heartbeater.cc:503] Master 127.14.142.62:42663 was elected leader, sending a full tablet report...
I20260926 21:23:40.065006 21542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/instance:
uuid: "5071e9a0f47249dc89c78bbb6724a0da"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "638f1eccadd8f53be1b8f51bb4bd0fe2"
server_key_iv: "39b4642cddb40252dd6e6a66948cadcc"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.066229 21542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal/instance:
uuid: "5071e9a0f47249dc89c78bbb6724a0da"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "638f1eccadd8f53be1b8f51bb4bd0fe2"
server_key_iv: "39b4642cddb40252dd6e6a66948cadcc"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.074381 21542 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
I20260926 21:23:40.079733 21557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:40.082343 21542 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260926 21:23:40.082542 21542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
uuid: "5071e9a0f47249dc89c78bbb6724a0da"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "638f1eccadd8f53be1b8f51bb4bd0fe2"
server_key_iv: "39b4642cddb40252dd6e6a66948cadcc"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.082759 21542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:40.102483 21542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:40.107059 21542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:40.107288 21542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:40.127739 21542 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.61:33303
I20260926 21:23:40.127746 21608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.61:33303 every 8 connection(s)
I20260926 21:23:40.129590 21542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
I20260926 21:23:40.133577 21609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:40.136106 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21542
I20260926 21:23:40.142463 21609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.61@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:40.162948 21614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.145488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.14.142.62:42663)
0926 21:23:40.146989 (+ 1501us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.146995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.147015 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:23:40.147078 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.147083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.147170 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.147313 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.149094 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.149813 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.151300 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.151707 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.151842 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.156248 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.156254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.156257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.156306 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.159599 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.160452 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.160459 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.160463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.160665 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.161097 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.161104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.161107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.161467 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.161645 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.162266 (+ 621us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.162531 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":1251,"threads_started":1}
I20260926 21:23:40.164134 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.138831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.14.142.61:33303)
0926 21:23:40.140002 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.140026 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.140119 (+ 93us) server_negotiation.cc:407] Connection header received
0926 21:23:40.142346 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.142377 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.142965 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.143505 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.145861 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.147880 (+ 2019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.149302 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.149985 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.150251 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.155999 (+ 5748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.156013 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.156034 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.156081 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.159770 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.160623 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.160630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.160632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.160783 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.161194 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.161199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.161207 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.161748 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.162073 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.162427 (+ 354us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.163005 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":275,"server-negotiator.queue_time_us":839,"thread_start_us":484,"threads_started":1}
I20260926 21:23:40.164186 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.145420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:53650)
0926 21:23:40.146281 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.146341 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.146521 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.147447 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.147453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.148642 (+ 1189us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.148925 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.148940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.149961 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.149966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.151140 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.151151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.151942 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.151949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.152102 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.152850 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.152878 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.155986 (+ 3108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.159791 (+ 3805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.159804 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.159820 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.160306 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.160781 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.160787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.160791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.160960 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.161622 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.161631 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.161844 (+ 213us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.162161 (+ 317us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.162779 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":319,"threads_started":1}
I20260926 21:23:40.167666 21609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.61@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
I20260926 21:23:40.183969 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.168768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.14.142.61:33303)
0926 21:23:40.168910 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.168916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.168934 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:23:40.169019 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.169025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.169086 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.169218 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.169875 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.170602 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.172136 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.172843 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.173034 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.176920 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.176930 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.176936 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.176976 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.180757 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.181552 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.181557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.181561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.181714 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.182247 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.182254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.182259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.182695 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.182892 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.183260 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.183575 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":52}
I20260926 21:23:40.184056 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.168490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47488)
0926 21:23:40.168768 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.168784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.168917 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.169231 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.169235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.169614 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.169741 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.169749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.170745 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.170750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.171912 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.171934 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.173044 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.173049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.173310 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.173962 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.173979 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.176497 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.180926 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.180933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.180939 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.181420 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.181826 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.181833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.181836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.181977 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.182845 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.182850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.183030 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.183185 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.183738 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":240}
I20260926 21:23:40.190021 21609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Bootstrap starting.
I20260926 21:23:40.193699 21609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:40.195235 21609 log.cc:824] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:40.198915 21609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: No bootstrap required, opened a new log
I20260926 21:23:40.207789 21609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } }
I20260926 21:23:40.208238 21609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:40.208351 21609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5071e9a0f47249dc89c78bbb6724a0da, State: Initialized, Role: FOLLOWER
I20260926 21:23:40.209030 21609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } }
I20260926 21:23:40.209954 21616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } } }
I20260926 21:23:40.210264 21616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:23:40.211045 21609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: configured and running, proceeding with master startup.
W20260926 21:23:40.221001 21627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:23:40.221163 21627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:23:40.231274 21542 master_runner.cc:428] Detected that this master 5071e9a0f47249dc89c78bbb6724a0da is joining an existing cluster
I20260926 21:23:40.231417 21542 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.142.61:33303
WARNING: no policy specified for oryx/127.14.142.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.60@KRBTEST.COM" created.
I20260926 21:23:40.261900 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.240755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47492 (local address 127.14.142.61:33303)
0926 21:23:40.240975 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.240987 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.245040 (+ 4053us) server_negotiation.cc:407] Connection header received
0926 21:23:40.245317 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.245322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.245403 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.245520 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.246977 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.248326 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.250530 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.251226 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.251325 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.254558 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.254564 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.254568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.254608 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.258584 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.259545 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.259550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.259553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.259723 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.260158 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.260163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.260166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.260546 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.260702 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.261246 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.261501 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":66}
I20260926 21:23:40.261881 21614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.239785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.14.142.62:42663)
0926 21:23:40.239992 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.240000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.245862 (+ 5862us) server_negotiation.cc:407] Connection header received
0926 21:23:40.246071 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.246081 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.246149 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.246305 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.247564 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.248660 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.250726 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.251406 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.251496 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.254599 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.254606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.254611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.254652 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.258742 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.259561 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.259566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.259570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.259716 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.260169 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.260174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.260177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.260547 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.260702 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.261268 (+ 566us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.261501 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:23:40.263069 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.239815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47492)
0926 21:23:40.244900 (+ 5085us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.244923 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.245140 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.245584 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.245591 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.246366 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.246808 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.246822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.248594 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.248599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.250344 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.250367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.251687 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.251695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.251902 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.252995 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.253019 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.254360 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.258762 (+ 4402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.258766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.258771 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.259412 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.259851 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.259858 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.259861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.260042 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.260770 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.260774 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.260967 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.262152 (+ 1185us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.262531 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4916,"mutex_wait_us":34,"spinlock_wait_cycles":50432,"thread_start_us":4735,"threads_started":1}
I20260926 21:23:40.263204 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.240312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:53664)
0926 21:23:40.245749 (+ 5437us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.245779 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.245899 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.246277 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.246284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.246967 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.247382 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.247395 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.248881 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.248887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.250572 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.250584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.251813 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.251818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.251975 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.252880 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.252907 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.254360 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.258912 (+ 4552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.258919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.258936 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.259412 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.259862 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.259866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.259869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.260042 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.260787 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.260790 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.261009 (+ 219us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.262152 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.262713 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5062,"mutex_wait_us":217,"spinlock_wait_cycles":3200,"thread_start_us":5226,"threads_started":1}
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.62@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.60.keytab.
Entry for principal oryx/127.14.142.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.60.keytab.
I20260926 21:23:40.284619 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:42243
--webserver_interface=127.14.142.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.60.keytab
--principal=oryx/127.14.142.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:23:40.286597 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.270079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.14.142.61:33303)
0926 21:23:40.270277 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.270284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.270569 (+ 285us) server_negotiation.cc:407] Connection header received
0926 21:23:40.270981 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.270987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.271096 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.271157 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.272461 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.273254 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.274766 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.275443 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.275605 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.279904 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.279910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.279913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.279950 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.283333 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.284164 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.284171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.284179 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.284318 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.284913 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.284916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.284918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.285208 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.285365 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.285914 (+ 549us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.286162 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:23:40.286700 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.269785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47506)
0926 21:23:40.270411 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.270478 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.270730 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.271399 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.271404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.272058 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.272285 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.272294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.273403 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.273411 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.274601 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.274615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.275649 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.275655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.275883 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.276696 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.276715 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.279628 (+ 2913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.283526 (+ 3898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.283543 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.283563 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.284025 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.284437 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.284442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.284446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.284685 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.285346 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.285355 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.285604 (+ 249us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.285801 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.286162 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":501,"threads_started":1}
I20260926 21:23:40.288920 21079 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.142.61:33303: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da"
member_type: NON_VOTER
last_known_addr {
host: "127.14.142.61"
port: 33303
}
attrs {
promote: true
}
}
dest_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46"
cas_config_opid_index: -1
I20260926 21:23:40.289865 21079 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } }
I20260926 21:23:40.301882 21584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:40.303929 21639 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5071e9a0f47249dc89c78bbb6724a0da (127.14.142.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } }
I20260926 21:23:40.306469 21584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Refusing update from remote peer b1e509ff7dc4430eb1417ebf8ed92b46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 21:23:40.316531 21643 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:40.317179 21639 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.142.61:33303
I20260926 21:23:40.318159 21542 master.cc:537] Master@127.14.142.61:33303 shutting down...
I20260926 21:23:40.319903 21642 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } }
I20260926 21:23:40.320544 21642 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
W20260926 21:23:40.323688 21063 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260926 21:23:40.326200 21063 consensus_peers.cc:487] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 -> Peer 5071e9a0f47249dc89c78bbb6724a0da (127.14.142.61:33303): Couldn't send request to peer 5071e9a0f47249dc89c78bbb6724a0da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 21:23:40.333606 21542 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:40.334120 21542 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:40.334367 21542 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: stopping tablet replica
I20260926 21:23:40.341315 21542 master.cc:552] Master@127.14.142.61:33303 shutdown complete.
I20260926 21:23:40.341440 21542 master_runner.cc:305] Clearing existing system tablet
I20260926 21:23:40.341841 21542 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:40.341971 21542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:40.344539 21542 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260926 21:23:40.346282 21647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:40.346650 21542 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:40.347081 21542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260926 21:23:40.347184 21542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
uuid: "5071e9a0f47249dc89c78bbb6724a0da"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "638f1eccadd8f53be1b8f51bb4bd0fe2"
server_key_iv: "39b4642cddb40252dd6e6a66948cadcc"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.351217 21542 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:40.356478 21542 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 21:23:40.356740 21542 log.cc:1196] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 21:23:40.357172 21542 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Deleting consensus metadata
I20260926 21:23:40.357519 21542 master_runner.cc:315] Copying system tablet from 127.14.142.62:42663
I20260926 21:23:40.361874 21542 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: tablet copy: Beginning tablet copy session from remote peer at address 127.14.142.62:42663
I20260926 21:23:40.378386 21614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.362770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53674 (local address 127.14.142.62:42663)
0926 21:23:40.363122 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.363132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.365354 (+ 2222us) server_negotiation.cc:407] Connection header received
0926 21:23:40.365662 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.365667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.365825 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.365935 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.367077 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.367879 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.369320 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.369811 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.369957 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.372632 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.372641 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.372646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.372695 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.375436 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.376249 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.376256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.376264 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.376391 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.376824 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.376829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.376833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.377137 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.377298 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.377723 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.378000 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:23:40.378415 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.362755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:53674)
0926 21:23:40.365178 (+ 2423us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.365202 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.365450 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.366138 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.366142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.366654 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.366914 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.366925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.368054 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.368060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.369148 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.369160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.369975 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.369981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.370188 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.371360 (+ 1172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.371386 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.372241 (+ 855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.375630 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.375637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.375641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.376101 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.376515 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.376521 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.376524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.376710 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.377264 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.377268 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.377449 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.377593 (+ 144us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.377933 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2175,"thread_start_us":1985,"threads_started":1}
I20260926 21:23:40.379402 21099 tablet_copy_service.cc:140] P b1e509ff7dc4430eb1417ebf8ed92b46: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5071e9a0f47249dc89c78bbb6724a0da ({username='oryx', principal='oryx/127.14.142.61@KRBTEST.COM'} at 127.0.0.1:53674)
I20260926 21:23:40.379695 21099 tablet_copy_service.cc:161] P b1e509ff7dc4430eb1417ebf8ed92b46: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5071e9a0f47249dc89c78bbb6724a0da at {username='oryx', principal='oryx/127.14.142.61@KRBTEST.COM'} at 127.0.0.1:53674: session id = 5071e9a0f47249dc89c78bbb6724a0da-00000000000000000000000000000000
I20260926 21:23:40.382845 21099 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 21:23:40.385097 21542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 21:23:40.390934 21542 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: tablet copy: Starting download of 0 data blocks...
I20260926 21:23:40.391330 21542 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: tablet copy: Starting download of 1 WAL segments...
I20260926 21:23:40.394261 21542 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 21:23:40.397841 21099 tablet_copy_service.cc:342] P b1e509ff7dc4430eb1417ebf8ed92b46: Request end of tablet copy session 5071e9a0f47249dc89c78bbb6724a0da-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.142.61@KRBTEST.COM'} at 127.0.0.1:53674
I20260926 21:23:40.398066 21099 tablet_copy_service.cc:433] P b1e509ff7dc4430eb1417ebf8ed92b46: ending tablet copy session 5071e9a0f47249dc89c78bbb6724a0da-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5071e9a0f47249dc89c78bbb6724a0da
I20260926 21:23:40.410979 21542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:40.413774 21654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:40.415005 21655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:40.423974 21657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:40.425828 21542 server_base.cc:1034] running on GCE node
I20260926 21:23:40.426185 21542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:40.426898 21542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:40.428115 21542 hybrid_clock.cc:648] HybridClock initialized: now 1790457820428040 us; error 46 us; skew 500 ppm
I20260926 21:23:40.428347 21542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:40.430620 21542 init.cc:377] Logged in from keytab as oryx/127.14.142.61@KRBTEST.COM (short username oryx)
I20260926 21:23:40.437690 21542 webserver.cc:460] Webserver started at http://127.14.142.61:34247/ using document root <none> and password file <none>
I20260926 21:23:40.438133 21542 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:40.438228 21542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:40.441895 21542 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260926 21:23:40.444181 21664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:40.444712 21542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260926 21:23:40.444886 21542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
uuid: "5071e9a0f47249dc89c78bbb6724a0da"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "638f1eccadd8f53be1b8f51bb4bd0fe2"
server_key_iv: "39b4642cddb40252dd6e6a66948cadcc"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.445073 21542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:40.468971 21542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:40.473322 21542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:40.519970 21542 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.61:33303
I20260926 21:23:40.519994 21715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.61:33303 every 8 connection(s)
I20260926 21:23:40.520958 21542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
I20260926 21:23:40.530541 21716 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:23:40.533560 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Bootstrap starting.
I20260926 21:23:40.550132 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:23:40.550666 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: Bootstrap complete.
I20260926 21:23:40.551838 21716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } }
I20260926 21:23:40.552076 21716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5071e9a0f47249dc89c78bbb6724a0da, State: Initialized, Role: LEARNER
I20260926 21:23:40.552294 21716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } }
I20260926 21:23:40.553184 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } }
I20260926 21:23:40.553504 21716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:40.553552 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: LEARNER
I20260926 21:23:40.574451 21730 catalog_manager.cc:1259] Loaded cluster ID: 1ccf6a148e6e4d4db595e57b7ebb5b99
I20260926 21:23:40.574587 21730 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: loading cluster ID for follower catalog manager: success
I20260926 21:23:40.577106 21730 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: acquiring CA information for follower catalog manager: success
I20260926 21:23:40.579015 21730 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:23:40.594051 21536 heartbeater.cc:503] Master 127.14.142.62:42663 was elected leader, sending a full tablet report...
W20260926 21:23:40.743629 21640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:40.744139 21640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:40.744354 21640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 21:23:40.747242 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.736930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47514)
0926 21:23:40.737217 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.737240 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.737369 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.738351 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.738356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.738389 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:40.738602 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.738612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.741588 (+ 2976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.741595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.743804 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.743823 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.746167 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.746174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.746391 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.746453 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.746572 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.746805 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260926 21:23:40.747301 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.737184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.14.142.61:33303)
0926 21:23:40.737929 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.737934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.737960 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:23:40.738031 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.738037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.738137 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.738281 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:40.738776 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.741336 (+ 2560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.744038 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.745928 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.746099 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.746416 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.746629 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.746824 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":410,"threads_started":1}
I20260926 21:23:40.748761 21691 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260926 21:23:40.749420 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e509ff7dc4430eb1417ebf8ed92b46. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } }
I20260926 21:23:40.749718 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: LEARNER
W20260926 21:23:40.757748 21640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:40.757936 21640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:40.758029 21640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:40.758136 21640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:40.758221 21640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:40.771500 21640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.60.keytab
--principal=oryx/127.14.142.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.142.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:40.773110 21640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:40.774509 21640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:40.790359 21736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:40.792204 21737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:40.805166 21739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:40.806033 21640 server_base.cc:1034] running on GCE node
I20260926 21:23:40.807017 21640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:40.808917 21640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:40.810189 21640 hybrid_clock.cc:648] HybridClock initialized: now 1790457820810123 us; error 53 us; skew 500 ppm
I20260926 21:23:40.810570 21640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:40.821154 21640 init.cc:377] Logged in from keytab as oryx/127.14.142.60@KRBTEST.COM (short username oryx)
I20260926 21:23:40.822976 21640 webserver.cc:460] Webserver started at http://127.14.142.60:39543/ using document root <none> and password file <none>
I20260926 21:23:40.823777 21640 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:40.823887 21640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:40.824141 21640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:40.827445 21640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/instance:
uuid: "ff4dca5cf61342328bfd100adbc94828"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "89a74c2340f4527b24adefe076eefe4b"
server_key_iv: "ff5214c0cf01730eb75237963e55a0d6"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.828198 21640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal/instance:
uuid: "ff4dca5cf61342328bfd100adbc94828"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "89a74c2340f4527b24adefe076eefe4b"
server_key_iv: "ff5214c0cf01730eb75237963e55a0d6"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.833981 21640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 21:23:40.837796 21746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:40.839313 21640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:40.839500 21640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
uuid: "ff4dca5cf61342328bfd100adbc94828"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "89a74c2340f4527b24adefe076eefe4b"
server_key_iv: "ff5214c0cf01730eb75237963e55a0d6"
server_key_version: "encryptionkey@0"
I20260926 21:23:40.839697 21640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:40.871090 21640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:40.875733 21640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:40.876051 21640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:40.896996 21640 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.60:42243
I20260926 21:23:40.897017 21797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.60:42243 every 8 connection(s)
I20260926 21:23:40.898555 21640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
I20260926 21:23:40.902371 21798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:40.908882 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21640
I20260926 21:23:40.911360 21798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.60@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.60@KRBTEST.COM
I20260926 21:23:40.930825 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.914428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53688 (local address 127.14.142.62:42663)
0926 21:23:40.915659 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.915664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.915681 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:23:40.916133 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.916138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.916223 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.916341 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.917745 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.918458 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.920100 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.920514 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.920723 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.924514 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.924519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.924522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.924610 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.927470 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.928249 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.928254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.928258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.928383 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.928922 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.928929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.928935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.929269 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.929481 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.930218 (+ 737us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.930462 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1136,"thread_start_us":1024,"threads_started":1}
I20260926 21:23:40.931849 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.914270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:53688)
0926 21:23:40.915086 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.915135 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.915864 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.916408 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.916414 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.917182 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.917575 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.917592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.918604 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.918610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.919948 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.919960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.920816 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.920823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.921063 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.921893 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.921963 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.924311 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.927608 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.927618 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.927632 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.928120 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.928510 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.928514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.928518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.928794 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.929405 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.929413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.929778 (+ 365us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.930127 (+ 349us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.930628 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":68,"thread_start_us":364,"threads_started":1}
I20260926 21:23:40.932623 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.911920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.14.142.60:42243)
0926 21:23:40.912876 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.912889 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.912931 (+ 42us) server_negotiation.cc:407] Connection header received
0926 21:23:40.915834 (+ 2903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.915859 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.916116 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.916358 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.918052 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.919397 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.920858 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.921550 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.921697 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.926490 (+ 4793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.926506 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.926520 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.926552 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.929609 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.930343 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.930349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.930351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.930484 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.930858 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.930863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.930865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.931306 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.931615 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.931932 (+ 317us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.932168 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":474,"threads_started":1}
I20260926 21:23:40.934223 21798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.60@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
I20260926 21:23:40.949630 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.935064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47516)
0926 21:23:40.935243 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.935263 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.935360 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.935615 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.935617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.935942 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.936072 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.936079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.937346 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.937353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.938591 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.938610 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.939689 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.939695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.939926 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.940818 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.940838 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.943352 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.946774 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.946779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.946782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.947243 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.947763 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.947767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.947769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.947914 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.948867 (+ 953us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.948871 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.949012 (+ 141us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.949136 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.949386 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 21:23:40.949796 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.935154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.14.142.61:33303)
0926 21:23:40.935390 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.935394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.935412 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:23:40.935461 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.935466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.935524 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.935595 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.936220 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.937081 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.938767 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.939441 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.939667 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.943661 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.943668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.943673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.943733 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.946610 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.947387 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.947393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.947396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.947614 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.948118 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.948124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.948127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.948732 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.948896 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.949273 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.949495 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260926 21:23:40.950850 21798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:23:40 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457820, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.60@KRBTEST.COM for oryx/127.14.142.60@KRBTEST.COM
I20260926 21:23:40.965096 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.951700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41770)
0926 21:23:40.951896 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:40.951916 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:40.952025 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:40.952413 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:40.952417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:40.953110 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:40.953339 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.953360 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.954525 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.954533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.955674 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:40.955688 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.956500 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.956505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:40.956797 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.957467 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:40.957485 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:40.959854 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:40.962828 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.962833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.962836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.963211 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.963561 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:40.963566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:40.963569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:40.963726 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:40.964241 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:40.964244 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:40.964358 (+ 114us) client_negotiation.cc:769] Sending connection context
0926 21:23:40.964507 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 21:23:40.964797 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 21:23:40.965154 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:40.951917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41770 (local address 127.14.142.60:42243)
0926 21:23:40.952074 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:40.952078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:40.952095 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:23:40.952135 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:40.952138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:40.952194 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:40.952281 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:40.953506 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.954224 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.955833 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:40.956299 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:40.956413 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:40.960098 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:40.960103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:40.960107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:40.960137 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:40.962645 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.963328 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.963332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.963334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.963441 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:40.963836 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:40.963840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:40.963842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:40.964144 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:40.964249 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:40.964667 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 21:23:40.964923 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":53}
I20260926 21:23:40.970197 21798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Bootstrap starting.
I20260926 21:23:40.973537 21798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:40.974591 21798 log.cc:824] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:40.977622 21798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: No bootstrap required, opened a new log
I20260926 21:23:40.986043 21798 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } }
I20260926 21:23:40.986699 21798 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:40.986904 21798 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff4dca5cf61342328bfd100adbc94828, State: Initialized, Role: FOLLOWER
I20260926 21:23:40.987543 21798 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } }
I20260926 21:23:40.988408 21805 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } } }
I20260926 21:23:40.988811 21805 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:23:40.989671 21798 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 21:23:40.998909 21816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:23:40.999102 21816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:23:41.004132 21640 master_runner.cc:428] Detected that this master ff4dca5cf61342328bfd100adbc94828 is joining an existing cluster
I20260926 21:23:41.004263 21640 master_runner.cc:432] Initiating AddMaster RPC to add 127.14.142.60:42243
I20260926 21:23:41.032809 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.009477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.14.142.62:42663)
0926 21:23:41.009656 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.009663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.009934 (+ 271us) server_negotiation.cc:407] Connection header received
0926 21:23:41.010204 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.010210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.010317 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.010410 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.012215 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.013752 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.016588 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.017476 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.017602 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.020827 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.020833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.020837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.020877 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.029339 (+ 8462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.030222 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.030228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.030231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.030395 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.030860 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.030866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.030880 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.031330 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.031478 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.032142 (+ 664us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.032405 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 21:23:41.034201 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.009027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:53696)
0926 21:23:41.009837 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.009856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.010045 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.010633 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.010638 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.011527 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.012004 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.012022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.014019 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.014027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.016341 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.016361 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.017981 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.017990 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.018267 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.019168 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.019202 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.020612 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.029530 (+ 8918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.029537 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.029544 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.030025 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.030543 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.030548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.030551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.030733 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.031554 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.031561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.031800 (+ 239us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.033228 (+ 1428us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.033621 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":640,"threads_started":1}
I20260926 21:23:41.036935 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.008993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.14.142.61:33303)
0926 21:23:41.009186 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.009193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.020377 (+ 11184us) server_negotiation.cc:407] Connection header received
0926 21:23:41.020817 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.020823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.020926 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.021007 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.022550 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.023644 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.025656 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.026345 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.026453 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.029955 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.029964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.029969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.030011 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.033686 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.034539 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.034547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.034550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.034715 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.035179 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.035183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.035186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.035551 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.035682 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.036332 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.036625 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 21:23:41.038690 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21140
I20260926 21:23:41.038808 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.009134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47524)
0926 21:23:41.020249 (+ 11115us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.020270 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.020668 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.021230 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.021234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.022020 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.022387 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.022398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.023844 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.023852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.025485 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.025498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.026786 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.026792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.027030 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.028266 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.028288 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.029711 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.033846 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.033852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.033856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.034402 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.034862 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.034866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.034869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.035050 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.035766 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.035771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.036026 (+ 255us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.037184 (+ 1158us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.038282 (+ 1098us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10949,"thread_start_us":18299,"threads_started":1}
I20260926 21:23:41.044117 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.010475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.14.142.60:42243)
0926 21:23:41.010649 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.010654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.027922 (+ 17268us) server_negotiation.cc:407] Connection header received
0926 21:23:41.028084 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.028090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.028166 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.028284 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.030039 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.030979 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.033286 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.034083 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.034181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.036669 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.036677 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.036682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.036714 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.040598 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.041394 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.041399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.041402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.041557 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.042020 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.042028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.042034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.042497 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.042689 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.043270 (+ 581us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.043592 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.62@KRBTEST.COM for oryx/127.14.142.60@KRBTEST.COM
I20260926 21:23:41.060168 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.010175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41784)
0926 21:23:41.027793 (+ 17618us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.027814 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.027963 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.028519 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.028524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.029436 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.029882 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.029894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.031150 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.031155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.032812 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.032827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.034494 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.034501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.034673 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.035316 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.035339 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.036472 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.040737 (+ 4265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.040743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.040746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.041259 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.041723 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.041730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.041736 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.041903 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.042770 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.042777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.042982 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.058456 (+ 15474us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.058889 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17455,"thread_start_us":10236,"threads_started":1}
I20260926 21:23:41.068866 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.040969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41786)
0926 21:23:41.041272 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.041292 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.041415 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.045005 (+ 3590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.045013 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.045649 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.045931 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.045946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.048333 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.048341 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.050329 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.050352 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.051914 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.051923 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.052191 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.053128 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.053162 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.056997 (+ 3835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.065125 (+ 8128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.065133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.065141 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.065849 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.066418 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.066426 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.066433 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.066620 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.067464 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.067472 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.067710 (+ 238us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.067869 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.068262 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260926 21:23:41.070571 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.041390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.14.142.60:42243)
0926 21:23:41.044486 (+ 3096us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.044495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.044516 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:23:41.044657 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.044664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.044776 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.046123 (+ 1347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.046406 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.048135 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.050646 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.051691 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.057247 (+ 5556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.057597 (+ 350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.057606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.057613 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.057657 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.064957 (+ 7300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.066059 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.066068 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.066074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.066252 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.066795 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.066802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.066808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.067278 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.069560 (+ 2282us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.069726 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.070049 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2952,"threads_started":1}
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.1@KRBTEST.COM
I20260926 21:23:41.074532 21079 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.14.142.60:42243: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ff4dca5cf61342328bfd100adbc94828"
member_type: NON_VOTER
last_known_addr {
host: "127.14.142.60"
port: 42243
}
attrs {
promote: true
}
}
dest_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46"
cas_config_opid_index: 5
I20260926 21:23:41.075452 21079 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } }
I20260926 21:23:41.077287 21642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ff4dca5cf61342328bfd100adbc94828 (127.14.142.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.081604 21691 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 LEARNER]: Refusing update from remote peer b1e509ff7dc4430eb1417ebf8ed92b46: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 21:23:41.084775 21642 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.14.142.60:42243
I20260926 21:23:41.085337 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.039850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.14.142.1:36175)
0926 21:23:41.061213 (+ 21363us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.061225 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.061260 (+ 35us) server_negotiation.cc:407] Connection header received
0926 21:23:41.061339 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.061348 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.061689 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.063967 (+ 2278us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.064234 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.065984 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.068035 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.069035 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.073063 (+ 4028us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.075891 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.075907 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.075926 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.075980 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.080043 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.081044 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.081053 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.081057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.081224 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.082380 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.082387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.082391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.082869 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.084432 (+ 1563us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.084635 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.084924 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21141,"thread_start_us":21995,"threads_started":1}
I20260926 21:23:41.086462 21640 master.cc:537] Master@127.14.142.60:42243 shutting down...
I20260926 21:23:41.087226 21263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:41.101364 21640 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:41.101925 21640 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:41.102182 21640 tablet_replica.cc:330] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: stopping tablet replica
I20260926 21:23:41.112408 21640 master.cc:552] Master@127.14.142.60:42243 shutdown complete.
I20260926 21:23:41.112525 21640 master_runner.cc:305] Clearing existing system tablet
I20260926 21:23:41.113014 21640 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:41.113135 21640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260926 21:23:41.113587 21064 consensus_peers.cc:487] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 -> Peer ff4dca5cf61342328bfd100adbc94828 (127.14.142.60:42243): Couldn't send request to peer ff4dca5cf61342328bfd100adbc94828. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260926 21:23:41.120630 21640 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.004s
I20260926 21:23:41.121011 21827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 21:23:41.130798 21833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:41.131207 21640 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:41.131505 21640 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.010s
I20260926 21:23:41.131608 21640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
uuid: "ff4dca5cf61342328bfd100adbc94828"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "89a74c2340f4527b24adefe076eefe4b"
server_key_iv: "ff5214c0cf01730eb75237963e55a0d6"
server_key_version: "encryptionkey@0"
I20260926 21:23:41.138396 21640 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:41.147392 21640 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 21:23:41.147557 21640 log.cc:1196] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 21:23:41.147298 21691 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ff4dca5cf61342328bfd100adbc94828 (127.14.142.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.148279 21640 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Deleting consensus metadata
I20260926 21:23:41.148589 21640 master_runner.cc:315] Copying system tablet from 127.14.142.62:42663
I20260926 21:23:41.157819 21827 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.158514 21827 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:41.160163 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: NON_VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: true } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.160565 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: LEARNER
I20260926 21:23:41.162091 21642 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: attempting to promote NON_VOTER 5071e9a0f47249dc89c78bbb6724a0da to VOTER
I20260926 21:23:41.163262 21642 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } }
I20260926 21:23:41.166709 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.165990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41800)
0926 21:23:41.166298 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.166485 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260926 21:23:41.168277 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.167697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41810)
0926 21:23:41.167945 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.168053 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260926 21:23:41.168656 21064 consensus_peers.cc:487] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 -> Peer ff4dca5cf61342328bfd100adbc94828 (127.14.142.60:42243): Couldn't send request to peer ff4dca5cf61342328bfd100adbc94828. Status: Network error: Client connection negotiation failed: client connection to 127.14.142.60:42243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 21:23:41.170217 21690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 LEARNER]: Refusing update from remote peer b1e509ff7dc4430eb1417ebf8ed92b46: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 21:23:41.171046 21642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260926 21:23:41.175230 21690 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5071e9a0f47249dc89c78bbb6724a0da (127.14.142.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.181226 21719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.181653 21719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:23:41.183485 21828 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5071e9a0f47249dc89c78bbb6724a0da (127.14.142.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.187256 21640 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: tablet copy: Beginning tablet copy session from remote peer at address 127.14.142.62:42663
I20260926 21:23:41.191800 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.192621 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:41.193265 21832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.193688 21832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:41.211854 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.188078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53710 (local address 127.14.142.62:42663)
0926 21:23:41.188328 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.188338 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.194662 (+ 6324us) server_negotiation.cc:407] Connection header received
0926 21:23:41.194881 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.194886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.194978 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.195090 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.195962 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.196841 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.198801 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.199486 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.199588 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.206341 (+ 6753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.206352 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.206356 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.206389 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.209180 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.209842 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.209849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.209852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.209989 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.210372 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.210378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.210382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.210670 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.210832 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.211217 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.211433 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 21:23:41.211983 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.188075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:53710)
0926 21:23:41.194516 (+ 6441us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.194540 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.194737 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.195123 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.195127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.195572 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.195804 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.195817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.197020 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.197025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.198640 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.198654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.199909 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.199916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.200164 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.205148 (+ 4984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.205167 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.206128 (+ 961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.209334 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.209340 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.209344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.209718 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.210100 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.210106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.210110 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.210262 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.210794 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.210798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.210978 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.211125 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.211436 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6276,"thread_start_us":6114,"threads_started":1}
I20260926 21:23:41.212893 21099 tablet_copy_service.cc:140] P b1e509ff7dc4430eb1417ebf8ed92b46: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ff4dca5cf61342328bfd100adbc94828 ({username='oryx', principal='oryx/127.14.142.60@KRBTEST.COM'} at 127.0.0.1:53710)
I20260926 21:23:41.213068 21099 tablet_copy_service.cc:161] P b1e509ff7dc4430eb1417ebf8ed92b46: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ff4dca5cf61342328bfd100adbc94828 at {username='oryx', principal='oryx/127.14.142.60@KRBTEST.COM'} at 127.0.0.1:53710: session id = ff4dca5cf61342328bfd100adbc94828-00000000000000000000000000000000
I20260926 21:23:41.215369 21099 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 21:23:41.217103 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 21:23:41.222759 21640 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: tablet copy: Starting download of 0 data blocks...
I20260926 21:23:41.223093 21640 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: tablet copy: Starting download of 1 WAL segments...
I20260926 21:23:41.225739 21640 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 21:23:41.229938 21099 tablet_copy_service.cc:342] P b1e509ff7dc4430eb1417ebf8ed92b46: Request end of tablet copy session ff4dca5cf61342328bfd100adbc94828-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.14.142.60@KRBTEST.COM'} at 127.0.0.1:53710
I20260926 21:23:41.230146 21099 tablet_copy_service.cc:433] P b1e509ff7dc4430eb1417ebf8ed92b46: ending tablet copy session ff4dca5cf61342328bfd100adbc94828-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ff4dca5cf61342328bfd100adbc94828
I20260926 21:23:41.241060 21640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:41.244002 21843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:41.250768 21844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:41.256482 21640 server_base.cc:1034] running on GCE node
W20260926 21:23:41.256951 21846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:41.257555 21640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:41.258344 21640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:41.259637 21640 hybrid_clock.cc:648] HybridClock initialized: now 1790457821259578 us; error 41 us; skew 500 ppm
I20260926 21:23:41.259910 21640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457821, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:41.262250 21640 init.cc:377] Logged in from keytab as oryx/127.14.142.60@KRBTEST.COM (short username oryx)
I20260926 21:23:41.262288 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21140
I20260926 21:23:41.270038 21640 webserver.cc:460] Webserver started at http://127.14.142.60:40585/ using document root <none> and password file <none>
I20260926 21:23:41.270498 21640 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:41.270617 21640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:41.275557 21640 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 21:23:41.278285 21853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:41.278839 21640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260926 21:23:41.279016 21640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
uuid: "ff4dca5cf61342328bfd100adbc94828"
format_stamp: "Formatted at 2026-09-26 21:23:40 on dist-test-slave-4d5t"
server_key: "89a74c2340f4527b24adefe076eefe4b"
server_key_iv: "ff5214c0cf01730eb75237963e55a0d6"
server_key_version: "encryptionkey@0"
I20260926 21:23:41.279268 21640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:41.291797 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:36175
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:42663
--webserver_port=40881
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:23:41.306963 21640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:41.312008 21640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:41.337941 21640 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.60:42243
I20260926 21:23:41.337998 21905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.60:42243 every 8 connection(s)
I20260926 21:23:41.338936 21640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
I20260926 21:23:41.344326 21906 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:23:41.348393 21906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Bootstrap starting.
I20260926 21:23:41.365875 21906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:23:41.366434 21906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Bootstrap complete.
I20260926 21:23:41.367588 21906 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } }
I20260926 21:23:41.367909 21906 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff4dca5cf61342328bfd100adbc94828, State: Initialized, Role: LEARNER
I20260926 21:23:41.368155 21906 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } }
I20260926 21:23:41.369051 21909 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.369383 21906 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:41.369554 21909 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: This master's current role is: LEARNER
I20260926 21:23:41.378935 21920 catalog_manager.cc:1259] Loaded cluster ID: 1ccf6a148e6e4d4db595e57b7ebb5b99
I20260926 21:23:41.379068 21920 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: loading cluster ID for follower catalog manager: success
I20260926 21:23:41.382493 21920 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: acquiring CA information for follower catalog manager: success
I20260926 21:23:41.385540 21920 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:23:41.660620 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.654002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41814)
0926 21:23:41.654241 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.654267 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.654420 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.655167 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.655172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.655200 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:41.655415 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.655425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.656864 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.656872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.658391 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.658405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.659679 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.659685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.659921 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.659967 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.660094 (+ 127us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.660254 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 21:23:41.660686 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.654202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41814 (local address 127.14.142.60:42243)
0926 21:23:41.654831 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.654836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.654852 (+ 16us) server_negotiation.cc:407] Connection header received
0926 21:23:41.654917 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.654921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.654998 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.655150 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:41.655595 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.656707 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.658554 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.659479 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.659654 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.659881 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.660121 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.660263 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":384,"threads_started":1}
I20260926 21:23:41.661937 21881 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 21:23:41.662606 21909 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1e509ff7dc4430eb1417ebf8ed92b46. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:41.662959 21909 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: This master's current role is: LEARNER
W20260926 21:23:41.682663 21858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:41.682943 21858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:41.683037 21858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:41.690593 21858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:41.690706 21858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:41.690821 21858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:41.701444 21858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:36175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=40881
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:41.702776 21858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:41.704066 21858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:41.718441 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:41.718502 21929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:41.718405 21926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:41.724851 21858 server_base.cc:1034] running on GCE node
I20260926 21:23:41.725584 21858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:41.726567 21858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:41.727833 21858 hybrid_clock.cc:648] HybridClock initialized: now 1790457821727779 us; error 47 us; skew 500 ppm
I20260926 21:23:41.728214 21858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457821, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:41.732481 21858 init.cc:377] Logged in from keytab as oryx/127.14.142.1@KRBTEST.COM (short username oryx)
I20260926 21:23:41.734484 21858 webserver.cc:460] Webserver started at http://127.14.142.1:40881/ using document root <none> and password file <none>
I20260926 21:23:41.735318 21858 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:41.735504 21858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:41.741873 21858 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 21:23:41.745371 21936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:41.747085 21858 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:41.747334 21858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "176d2d568df44be4b8bfe5ee05793052"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "7317870f27fd4357a55fb42ff7a3b9c4"
server_key_iv: "5ae79a71ea1cb5e7a996fdff9c421d61"
server_key_version: "encryptionkey@0"
I20260926 21:23:41.748608 21858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:41.766495 21858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:41.771202 21858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:41.771535 21858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:41.773018 21858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:41.774717 21858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:41.774819 21858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:41.774909 21858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:41.774977 21858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:41.816468 21858 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:36175
I20260926 21:23:41.816521 22048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:36175 every 8 connection(s)
I20260926 21:23:41.818166 21858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:41.822891 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 21858
I20260926 21:23:41.823287 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21274
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457821, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.60@KRBTEST.COM
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.2@KRBTEST.COM
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457821, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
Sep 26 21:23:41 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457821, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:41.854166 21921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.825480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:58823 (local address 127.14.142.60:42243)
0926 21:23:41.825675 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.825682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.831053 (+ 5371us) server_negotiation.cc:407] Connection header received
0926 21:23:41.831320 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.831328 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.831419 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.831503 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.834346 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.835453 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.838009 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.838634 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.838735 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.844300 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.844308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.844313 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.844358 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.848389 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.849766 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.849776 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.849781 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.849951 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.850446 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.850451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.850455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.850799 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.850955 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.853220 (+ 2265us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.853667 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:23:41.857028 22058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.825361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.14.142.1:58823)
0926 21:23:41.830909 (+ 5548us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.830945 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.831165 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.832039 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.832054 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.833137 (+ 1083us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.834104 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.834133 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.835667 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.835675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.837818 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.837834 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.839101 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.839108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.839402 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.840273 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.840315 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.844008 (+ 3693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.848604 (+ 4596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.848615 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.848642 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.849585 (+ 943us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.850082 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.850085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.850088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.850298 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.850998 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.851014 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.852682 (+ 1668us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.854615 (+ 1933us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.855299 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5404,"thread_start_us":5251,"threads_started":1}
I20260926 21:23:41.859514 22049 heartbeater.cc:347] Connected to a master server at 127.14.142.60:42243
I20260926 21:23:41.859906 22049 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:41.861054 22049 heartbeater.cc:511] Master 127.14.142.60:42243 requested a full tablet report, sending...
I20260926 21:23:41.861852 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.822093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:43057)
0926 21:23:41.825060 (+ 2967us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.825085 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.830267 (+ 5182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.841108 (+ 10841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.841113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.841543 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.842001 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.842027 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.843587 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.843592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.845713 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.845729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.847523 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.847529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.847768 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.848480 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.848515 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.852243 (+ 3728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.857438 (+ 5195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.857444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.857448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.858721 (+ 1273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.859371 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.859377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.859381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.859654 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.860421 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.860427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.860680 (+ 253us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.860840 (+ 160us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.861271 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2800,"mutex_wait_us":543,"thread_start_us":1486,"threads_started":1}
I20260926 21:23:41.862159 22059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.825793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.14.142.2:41363)
0926 21:23:41.831338 (+ 5545us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.831347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.832113 (+ 766us) server_negotiation.cc:407] Connection header received
0926 21:23:41.832183 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.832192 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.832499 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.832687 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.834952 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.836485 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.838371 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.839254 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.839372 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.849500 (+ 10128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.849529 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.849557 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.849614 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.856732 (+ 7118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.858054 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.858064 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.858068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.858264 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.858769 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.858774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.858777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.859319 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.859691 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.861338 (+ 1647us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.861703 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5348,"thread_start_us":5187,"threads_started":1}
I20260926 21:23:41.862488 22056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.822093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.14.142.1:41329)
0926 21:23:41.824426 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:41.824486 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:41.829550 (+ 5064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:41.840375 (+ 10825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:41.840383 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:41.841135 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:41.841621 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.841637 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.843544 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.843550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.845819 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:41.845836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.846939 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.846947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:41.847141 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.848198 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:41.848228 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:41.851433 (+ 3205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:41.857704 (+ 6271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.857713 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.857719 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.858869 (+ 1150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.859407 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:41.859412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:41.859415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:41.859756 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:41.860525 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:41.860531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:41.860756 (+ 225us) client_negotiation.cc:769] Sending connection context
0926 21:23:41.860890 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 21:23:41.861465 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2009,"mutex_wait_us":235,"thread_start_us":3278,"threads_started":1}
I20260926 21:23:41.862991 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.821465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:43057 (local address 127.14.142.62:42663)
0926 21:23:41.840807 (+ 19342us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.840813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.840834 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:23:41.840894 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.840900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.840970 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.841083 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.842209 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.843365 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.846031 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.846884 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.847383 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.852662 (+ 5279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.852670 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.852674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.852728 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.857241 (+ 4513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.858975 (+ 1734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.858981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.858985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.859223 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.859824 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.859832 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.859837 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.860259 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.862124 (+ 1865us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.862314 (+ 190us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.862620 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19223,"thread_start_us":19087,"threads_started":1}
I20260926 21:23:41.864205 21397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:41.864626 22050 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42663
I20260926 21:23:41.864786 22050 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:41.864868 21871 ts_manager.cc:194] Registered new tserver with Master: 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175)
I20260926 21:23:41.865325 22050 heartbeater.cc:511] Master 127.14.142.62:42663 requested a full tablet report, sending...
I20260926 21:23:41.866550 22054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:41.821443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:41329 (local address 127.14.142.61:33303)
0926 21:23:41.839940 (+ 18497us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:41.839946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:41.839973 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:41.840045 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:41.840051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:41.840160 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:41.840288 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:41.842209 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.843357 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.845958 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:41.846713 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:41.847839 (+ 1126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:41.851681 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:41.851687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:41.851691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:41.851728 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:41.857059 (+ 5331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.859048 (+ 1989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.859054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.859058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.859252 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:41.859890 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:41.859895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:41.859898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:41.860379 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:41.862941 (+ 2562us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:41.863091 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 21:23:41.865837 (+ 2746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18343,"thread_start_us":18232,"threads_started":1}
I20260926 21:23:41.866994 21079 ts_manager.cc:194] Re-registered known tserver with Master: 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175)
I20260926 21:23:41.867980 22051 heartbeater.cc:347] Connected to a master server at 127.14.142.61:33303
I20260926 21:23:41.868139 22051 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:41.868726 22051 heartbeater.cc:511] Master 127.14.142.61:33303 requested a full tablet report, sending...
I20260926 21:23:41.868698 21079 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:43057
I20260926 21:23:41.871346 21681 ts_manager.cc:194] Registered new tserver with Master: 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175)
I20260926 21:23:41.952176 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21274
I20260926 21:23:41.969350 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:41363
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:42663
--webserver_port=42713
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:23:42.177749 21827 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: NON_VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: true } } }
I20260926 21:23:42.178158 21827 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:42.180079 22062 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: attempting to promote NON_VOTER ff4dca5cf61342328bfd100adbc94828 to VOTER
I20260926 21:23:42.180743 22062 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } }
I20260926 21:23:42.182679 21690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Refusing update from remote peer b1e509ff7dc4430eb1417ebf8ed92b46: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 21:23:42.182880 21881 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 LEARNER]: Refusing update from remote peer b1e509ff7dc4430eb1417ebf8ed92b46: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 21:23:42.183333 21827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 21:23:42.183714 22063 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 21:23:42.186923 22063 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ff4dca5cf61342328bfd100adbc94828 (127.14.142.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.187498 21690 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ff4dca5cf61342328bfd100adbc94828 (127.14.142.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.190663 22063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.190984 22063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:42.191007 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.191339 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:42.193271 22067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.193745 22067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:23:42.193727 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.193974 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:42.194465 21881 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ff4dca5cf61342328bfd100adbc94828 (127.14.142.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.199390 22068 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:42.199873 22068 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 21:23:42.331789 22061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:42.332090 22061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:42.332254 22061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:42.339758 22061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:42.339885 22061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:42.340077 22061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:42.350373 22061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:41363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=42713
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:42.351778 22061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:42.353242 22061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:42.370817 22074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:42.371021 22073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:42.370982 22076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:42.372792 22061 server_base.cc:1034] running on GCE node
I20260926 21:23:42.373805 22061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:42.375260 22061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:42.376641 22061 hybrid_clock.cc:648] HybridClock initialized: now 1790457822376503 us; error 50 us; skew 500 ppm
I20260926 21:23:42.377125 22061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:42 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457822, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:42.382221 22061 init.cc:377] Logged in from keytab as oryx/127.14.142.2@KRBTEST.COM (short username oryx)
I20260926 21:23:42.384094 22061 webserver.cc:460] Webserver started at http://127.14.142.2:42713/ using document root <none> and password file <none>
I20260926 21:23:42.384912 22061 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:42.385043 22061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:42.391037 22061 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:23:42.393994 22083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:42.395702 22061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:23:42.395879 22061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "d16ca4cc2861428faad7658e6f5f7445"
format_stamp: "Formatted at 2026-09-26 21:23:38 on dist-test-slave-4d5t"
server_key: "4d433922a55bbd9d2a0b3d2d374e242d"
server_key_iv: "929b24540aa1641378ae3b735b686e64"
server_key_version: "encryptionkey@0"
I20260926 21:23:42.397099 22061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:42.418653 22061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:42.423550 22061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:42.423823 22061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:42.424839 22061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:42.426417 22061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:42.426515 22061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:42.426589 22061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:42.426636 22061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:42.463191 22061 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:41363
I20260926 21:23:42.463305 22195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:41363 every 8 connection(s)
I20260926 21:23:42.464746 22061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:42.474400 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 22061
I20260926 21:23:42.474756 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21408
Sep 26 21:23:42 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457822, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.60@KRBTEST.COM
Sep 26 21:23:42 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457822, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
Sep 26 21:23:42 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457822, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:42.499629 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.467749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:48829 (local address 127.14.142.61:33303)
0926 21:23:42.468242 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:42.468249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:42.469506 (+ 1257us) server_negotiation.cc:407] Connection header received
0926 21:23:42.472684 (+ 3178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:42.472693 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:42.472804 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:42.472916 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:42.477499 (+ 4583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.479079 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.481946 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.482578 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.482701 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.490360 (+ 7659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:42.490371 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:42.490376 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:42.490435 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:42.495301 (+ 4866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.496281 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.496287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.496291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.496439 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.497002 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.497006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.497009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.497403 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:42.497559 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:42.499019 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0926 21:23:42.499255 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":685,"threads_started":1}
I20260926 21:23:42.500857 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.468399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.14.142.2:48829)
0926 21:23:42.469306 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:42.469347 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:42.472179 (+ 2832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:42.473065 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:42.473072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:42.474073 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:42.477276 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:42.477295 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.479334 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.479342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:42.481712 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:42.481727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.482981 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.482991 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:42.483238 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.484620 (+ 1382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:42.484654 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:42.490023 (+ 5369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:42.495476 (+ 5453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:42.495486 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:42.495504 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:42.496125 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:42.496601 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:42.496606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:42.496610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:42.496877 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:42.497635 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:42.497644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:42.498167 (+ 523us) client_negotiation.cc:769] Sending connection context
0926 21:23:42.498486 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 21:23:42.500102 (+ 1616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":138,"spinlock_wait_cycles":22016,"thread_start_us":360,"threads_started":1}
I20260926 21:23:42.501276 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.468399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.14.142.2:50275)
0926 21:23:42.470823 (+ 2424us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:42.470845 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:42.472217 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:42.473008 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:42.473019 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:42.474073 (+ 1054us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:42.475361 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:42.475386 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.477561 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.477571 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:42.480971 (+ 3400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:42.480999 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.482703 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.482714 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:42.483209 (+ 495us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.484452 (+ 1243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:42.484500 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:42.491833 (+ 7333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:42.496218 (+ 4385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:42.496226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:42.496234 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:42.496937 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:42.497429 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:42.497436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:42.497441 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:42.497638 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:42.498404 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:42.498409 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:42.498590 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 21:23:42.498770 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 21:23:42.499239 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2254,"mutex_wait_us":271,"spinlock_wait_cycles":87808,"thread_start_us":1853,"threads_started":1}
I20260926 21:23:42.501847 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.471166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:42963 (local address 127.14.142.62:42663)
0926 21:23:42.472366 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:42.472373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:42.472396 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:23:42.472625 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:42.472630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:42.472729 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:42.472834 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:42.475620 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.477020 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.486348 (+ 9328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.487349 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.487515 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.493943 (+ 6428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:42.493952 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:42.493960 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:42.494021 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:42.498060 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.498947 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.498952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.498956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.499133 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.499597 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.499602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.499605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.499964 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:42.500118 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:42.500950 (+ 832us) server_negotiation.cc:299] Negotiation successful
0926 21:23:42.501231 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1071,"thread_start_us":918,"threads_started":1}
I20260926 21:23:42.502519 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.467736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:50275 (local address 127.14.142.60:42243)
0926 21:23:42.468952 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:42.468960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:42.470941 (+ 1981us) server_negotiation.cc:407] Connection header received
0926 21:23:42.472511 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:42.472518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:42.472644 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:42.472761 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:42.475584 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.477287 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.481277 (+ 3990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.482336 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.483921 (+ 1585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.492052 (+ 8131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:42.492058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:42.492061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:42.492097 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:42.496037 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.497121 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.497127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.497135 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.497283 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.497865 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.497872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.497877 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.498258 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:42.500862 (+ 2604us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:42.501007 (+ 145us) server_negotiation.cc:299] Negotiation successful
0926 21:23:42.502091 (+ 1084us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1052,"thread_start_us":960,"threads_started":1}
I20260926 21:23:42.502977 22198 heartbeater.cc:347] Connected to a master server at 127.14.142.61:33303
I20260926 21:23:42.503257 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.471139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:42963)
0926 21:23:42.471819 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:42.471861 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:42.472465 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:42.473271 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:42.473274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:42.474073 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:42.475339 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:42.475373 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.481637 (+ 6264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.481643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:42.486136 (+ 4493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:42.486159 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.487714 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.487725 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:42.488096 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.489848 (+ 1752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:42.489887 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:42.493703 (+ 3816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:42.498229 (+ 4526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:42.498235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:42.498239 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:42.498806 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:42.499279 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:42.499283 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:42.499287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:42.499456 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:42.500193 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:42.500198 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:42.500385 (+ 187us) client_negotiation.cc:769] Sending connection context
0926 21:23:42.500540 (+ 155us) client_negotiation.cc:240] Negotiation successful
0926 21:23:42.502452 (+ 1912us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":1291,"spinlock_wait_cycles":3840,"thread_start_us":406,"threads_started":1}
I20260926 21:23:42.503386 22198 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:42.504267 22197 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42663
I20260926 21:23:42.504422 22197 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:42.504504 22198 heartbeater.cc:511] Master 127.14.142.61:33303 requested a full tablet report, sending...
I20260926 21:23:42.504990 22197 heartbeater.cc:511] Master 127.14.142.62:42663 requested a full tablet report, sending...
I20260926 21:23:42.505167 22196 heartbeater.cc:347] Connected to a master server at 127.14.142.60:42243
I20260926 21:23:42.505292 22196 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:42.505805 22196 heartbeater.cc:511] Master 127.14.142.60:42243 requested a full tablet report, sending...
Sep 26 21:23:42 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.3@KRBTEST.COM
I20260926 21:23:42.506371 21079 ts_manager.cc:194] Re-registered known tserver with Master: d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363)
I20260926 21:23:42.506589 21681 ts_manager.cc:194] Registered new tserver with Master: d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363)
I20260926 21:23:42.507666 21079 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:42963
I20260926 21:23:42.508965 21871 ts_manager.cc:194] Registered new tserver with Master: d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363)
I20260926 21:23:42.514601 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:42.475781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55350 (local address 127.14.142.3:37659)
0926 21:23:42.490886 (+ 15105us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:42.490895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:42.491713 (+ 818us) server_negotiation.cc:407] Connection header received
0926 21:23:42.492681 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:42.492692 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:42.493000 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:42.493163 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:42.495493 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.497059 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.499063 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:42.500279 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:42.500395 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:42.507446 (+ 7051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:42.507459 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:42.507473 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:42.507514 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:42.511152 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.511903 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.511911 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.511913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.512073 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:42.512975 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:42.512980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:42.512983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:42.513340 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:42.513650 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:42.514035 (+ 385us) server_negotiation.cc:299] Negotiation successful
0926 21:23:42.514266 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14898,"thread_start_us":15104,"threads_started":1}
I20260926 21:23:42.516027 21531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:42.614177 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21408
W20260926 21:23:42.632496 21065 connection.cc:535] server connection from 127.14.142.3:58877 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 21:23:42.633169 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:37659
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:42663
--webserver_port=44331
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:23:42.872445 22050 heartbeater.cc:503] Master 127.14.142.62:42663 was elected leader, sending a full tablet report...
W20260926 21:23:42.991303 22209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:42.991601 22209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:42.991806 22209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:42.999609 22209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:42.999748 22209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:42.999933 22209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:43.011260 22209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:37659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=44331
--tserver_master_addrs=127.14.142.62:42663,127.14.142.61:33303,127.14.142.60:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:43.012859 22209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:43.014293 22209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:43.029170 22215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:43.029172 22217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:43.029742 22214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:43.030476 22209 server_base.cc:1034] running on GCE node
I20260926 21:23:43.031570 22209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:43.032851 22209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:43.034124 22209 hybrid_clock.cc:648] HybridClock initialized: now 1790457823034057 us; error 61 us; skew 500 ppm
I20260926 21:23:43.034487 22209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:43 dist-test-slave-4d5t krb5kdc[21034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457823, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:43.038012 22209 init.cc:377] Logged in from keytab as oryx/127.14.142.3@KRBTEST.COM (short username oryx)
I20260926 21:23:43.039729 22209 webserver.cc:460] Webserver started at http://127.14.142.3:44331/ using document root <none> and password file <none>
I20260926 21:23:43.040436 22209 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:43.040614 22209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:43.047226 22209 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:43.050288 22224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:43.052361 22209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260926 21:23:43.052590 22209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "25bb5e784e894167804079ec5800aa57"
format_stamp: "Formatted at 2026-09-26 21:23:39 on dist-test-slave-4d5t"
server_key: "c04697199dde06911b87fb3832267795"
server_key_iv: "7ad3a400a9a00502ffc7ce29b113e332"
server_key_version: "encryptionkey@0"
I20260926 21:23:43.053874 22209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:43.087512 22209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:43.092855 22209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:43.093196 22209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:43.094209 22209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:43.095692 22209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:43.095824 22209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:43.095963 22209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:43.096060 22209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:43.132917 22209 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:37659
I20260926 21:23:43.133006 22336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:37659 every 8 connection(s)
I20260926 21:23:43.134443 22209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:43.137878 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 22209
Sep 26 21:23:43 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457823, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.60@KRBTEST.COM
Sep 26 21:23:43 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457823, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.61@KRBTEST.COM
I20260926 21:23:43.201463 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.138852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:42993 (local address 127.14.142.60:42243)
0926 21:23:43.160843 (+ 21991us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.160854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.164208 (+ 3354us) server_negotiation.cc:407] Connection header received
0926 21:23:43.168456 (+ 4248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.168467 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.168641 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.168755 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.175853 (+ 7098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.177283 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.179284 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.180150 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.180282 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.191743 (+ 11461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.191751 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.191759 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.191809 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.196901 (+ 5092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.197992 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.197999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.198004 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.198165 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.198831 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.198837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.198843 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.199261 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.199403 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.200625 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.200881 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21804,"thread_start_us":22001,"threads_started":1}
I20260926 21:23:43.204401 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.138119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.14.142.3:42993)
0926 21:23:43.161823 (+ 23704us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.161932 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.167952 (+ 6020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.170533 (+ 2581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.170555 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.171737 (+ 1182us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:43.175613 (+ 3876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.175630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.177487 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.177491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.179096 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.179109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.182557 (+ 3448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.182569 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.182941 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.185234 (+ 2293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:43.185268 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:43.190198 (+ 4930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:43.197077 (+ 6879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:43.197088 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:43.197111 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:43.197839 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:43.198347 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:43.198353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:43.198360 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:43.198676 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:43.199539 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:43.199551 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:43.200086 (+ 535us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.201785 (+ 1699us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.202581 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23244,"thread_start_us":31273,"threads_started":1}
I20260926 21:23:43.205510 22350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.154596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47530 (local address 127.14.142.61:33303)
0926 21:23:43.155284 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.155294 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.166335 (+ 11041us) server_negotiation.cc:407] Connection header received
0926 21:23:43.167414 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.167424 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.167539 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.167641 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.174637 (+ 6996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.176051 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.180510 (+ 4459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.181326 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.181466 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.190548 (+ 9082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.190557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.190564 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.190611 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.199890 (+ 9279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.201127 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.201136 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.201141 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.201384 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.202077 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.202085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.202094 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.202574 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.202869 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.204740 (+ 1871us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.205013 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":497,"thread_start_us":323,"threads_started":1}
I20260926 21:23:43.206419 22337 heartbeater.cc:347] Connected to a master server at 127.14.142.60:42243
I20260926 21:23:43.206784 22337 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:43.207803 22337 heartbeater.cc:511] Master 127.14.142.60:42243 requested a full tablet report, sending...
I20260926 21:23:43.209231 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.137475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:51971 (local address 127.14.142.61:33303)
0926 21:23:43.150600 (+ 13125us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.150605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.170094 (+ 19489us) server_negotiation.cc:407] Connection header received
0926 21:23:43.172366 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.172376 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.172479 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.172638 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.179365 (+ 6727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.180832 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.183610 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.184419 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.184579 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.192320 (+ 7741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.192327 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.192332 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.192384 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.200174 (+ 7790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.201195 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.201206 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.201209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.201462 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.202034 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.202040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.202043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.203086 (+ 1043us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.203229 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.208442 (+ 5213us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.208750 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1321,"server-negotiator.queue_time_us":12986,"thread_start_us":12864,"threads_started":1}
I20260926 21:23:43.209941 21871 ts_manager.cc:194] Registered new tserver with Master: 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
Sep 26 21:23:43 dist-test-slave-4d5t krb5kdc[21034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457823, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:43.211565 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.138119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.14.142.3:51971)
0926 21:23:43.169851 (+ 31732us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.169872 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.172132 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.172969 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.172977 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.173869 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:43.175065 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.175103 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.181035 (+ 5932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.181041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.183113 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.183127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.184945 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.184954 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.185233 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.185925 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:43.185949 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:43.192064 (+ 6115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:43.200438 (+ 8374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:43.200445 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:43.200449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:43.201043 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:43.201693 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:43.201698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:43.201702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:43.201885 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:43.207740 (+ 5855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:43.207748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:43.208045 (+ 297us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.209727 (+ 1682us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.211037 (+ 1310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31556,"mutex_wait_us":48,"spinlock_wait_cycles":88960,"thread_start_us":26028,"threads_started":1}
I20260926 21:23:43.212811 22339 heartbeater.cc:347] Connected to a master server at 127.14.142.61:33303
I20260926 21:23:43.212958 22339 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:43.213452 22339 heartbeater.cc:511] Master 127.14.142.61:33303 requested a full tablet report, sending...
I20260926 21:23:43.214578 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.165770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.14.142.60:42243)
0926 21:23:43.181310 (+ 15540us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.181319 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.181344 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:23:43.181418 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.181426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.181542 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.181680 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.187811 (+ 6131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.189289 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.195525 (+ 6236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.196325 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.196468 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.200377 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.200386 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.200392 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.200436 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.205410 (+ 4974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.206639 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.206645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.206650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.206823 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.211842 (+ 5019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.211848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.211854 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.212300 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.212553 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.213879 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.214190 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15342,"thread_start_us":16085,"threads_started":1}
I20260926 21:23:43.214720 21681 ts_manager.cc:194] Registered new tserver with Master: 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:43.218387 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.154518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.14.142.62:42663)
0926 21:23:43.185397 (+ 30879us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.185403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.185428 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:23:43.185492 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.185500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.185596 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.186382 (+ 786us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.190031 (+ 3649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.192648 (+ 2617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.195115 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.195894 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.196795 (+ 901us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.200041 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.200051 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.200059 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.200108 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.215321 (+ 15213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.216041 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.216047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.216051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.216208 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.216692 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.216698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.216701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.217051 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.217344 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.217734 (+ 390us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.217993 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30756,"thread_start_us":30649,"threads_started":1}
I20260926 21:23:43.220491 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.185379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:57951 (local address 127.14.142.62:42663)
0926 21:23:43.192016 (+ 6637us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.192023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.192045 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:23:43.192101 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.192106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.192193 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.192317 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.197398 (+ 5081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.204316 (+ 6918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.206566 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.207304 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.208147 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.211980 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.211992 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.212001 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.212066 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.217150 (+ 5084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.217895 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.217899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.217901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.218008 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.218516 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.218526 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.218530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.218816 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.218985 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.219961 (+ 976us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.220138 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6492,"thread_start_us":7101,"threads_started":1}
I20260926 21:23:43.220732 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.170113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:57951)
0926 21:23:43.176747 (+ 6634us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.176769 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.176889 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.193761 (+ 16872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.193768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.194437 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:43.194827 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.194844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.204521 (+ 9677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.204528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.206323 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.206341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.207545 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.207553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.207783 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.208485 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:43.208509 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:43.211712 (+ 3203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:43.217320 (+ 5608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:43.217326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:43.217330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:43.217755 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:43.218163 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:43.218169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:43.218175 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:43.218373 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:43.218963 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:43.218968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:43.219464 (+ 496us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.219885 (+ 421us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.220260 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6493,"thread_start_us":6845,"threads_started":1}
I20260926 21:23:43.222117 22338 heartbeater.cc:347] Connected to a master server at 127.14.142.62:42663
I20260926 21:23:43.222295 22338 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:43.222893 22338 heartbeater.cc:511] Master 127.14.142.62:42663 requested a full tablet report, sending...
I20260926 21:23:43.224023 21078 ts_manager.cc:194] Re-registered known tserver with Master: 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:43.225211 21078 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:57951
I20260926 21:23:43.223445 21079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:43.231268 21079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:43.288295 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.253725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.14.142.1:36175)
0926 21:23:43.280159 (+ 26434us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.280170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.280211 (+ 41us) server_negotiation.cc:407] Connection header received
0926 21:23:43.280364 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.280378 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.280801 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.281852 (+ 1051us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.282081 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.283676 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.285892 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.287387 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.287552 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.287709 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.287953 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.288067 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26221,"thread_start_us":25998,"threads_started":1}
I20260926 21:23:43.288411 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.253473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36175 (local address 127.0.0.1:42492)
0926 21:23:43.260973 (+ 7500us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.261015 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.261208 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.281156 (+ 19948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.281161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.281192 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.281591 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.281601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.283798 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.283805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.285737 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.285749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.287606 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.287613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.287772 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.287806 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.287907 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.288045 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":1108,"threads_started":1}
I20260926 21:23:43.289947 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.254941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41363 (local address 127.0.0.1:41898)
0926 21:23:43.282427 (+ 27486us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.282441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.282530 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.283086 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.283089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.283111 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.283505 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.283517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.285910 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.285914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.287330 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.287341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.289116 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.289122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.289298 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.289335 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.289433 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.289573 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27379,"mutex_wait_us":24,"thread_start_us":27226,"threads_started":1}
I20260926 21:23:43.291647 22371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.254611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.14.142.3:37659)
0926 21:23:43.279496 (+ 24885us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.279507 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.281703 (+ 2196us) server_negotiation.cc:407] Connection header received
0926 21:23:43.281929 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.281937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.282157 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.282354 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.283088 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.284882 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.289367 (+ 4485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.290690 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.290791 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.290916 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.291288 (+ 372us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.291407 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24649,"thread_start_us":26045,"threads_started":1}
I20260926 21:23:43.291769 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.254319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:37659 (local address 127.0.0.1:55360)
0926 21:23:43.281478 (+ 27159us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.281506 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.281744 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.282632 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.282635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.282649 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.282910 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.282917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.284998 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.285004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.287315 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.287332 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.290905 (+ 3573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.290912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.291097 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.291131 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.291246 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.291379 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26948,"mutex_wait_us":26,"thread_start_us":26758,"threads_started":1}
I20260926 21:23:43.292666 21984 tablet_service.cc:1467] Processing CreateTablet for tablet 31cfbd46870847f08f235940db3f90c7 (DEFAULT_TABLE table=test-table [id=a355a067656b4d31a07e5a4774ee69e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:43.294224 21984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31cfbd46870847f08f235940db3f90c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:43.295164 22272 tablet_service.cc:1467] Processing CreateTablet for tablet 31cfbd46870847f08f235940db3f90c7 (DEFAULT_TABLE table=test-table [id=a355a067656b4d31a07e5a4774ee69e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:43.296200 22272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31cfbd46870847f08f235940db3f90c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:43.301170 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.255217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.14.142.2:41363)
0926 21:23:43.278608 (+ 23391us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.278623 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.282531 (+ 3908us) server_negotiation.cc:407] Connection header received
0926 21:23:43.282666 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.282672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.282879 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.283053 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.283735 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.284780 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.287491 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.288918 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.289075 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.289273 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.300701 (+ 11428us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.300873 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23064,"thread_start_us":23421,"threads_started":1}
I20260926 21:23:43.305585 22131 tablet_service.cc:1467] Processing CreateTablet for tablet 31cfbd46870847f08f235940db3f90c7 (DEFAULT_TABLE table=test-table [id=a355a067656b4d31a07e5a4774ee69e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:43.307107 22131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31cfbd46870847f08f235940db3f90c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:43.308493 22374 tablet_bootstrap.cc:492] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Bootstrap starting.
I20260926 21:23:43.311035 22375 tablet_bootstrap.cc:492] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Bootstrap starting.
I20260926 21:23:43.313674 22374 tablet_bootstrap.cc:654] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:43.314841 22375 tablet_bootstrap.cc:654] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:43.315153 22374 log.cc:824] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:43.316212 22375 log.cc:824] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:43.318640 22374 tablet_bootstrap.cc:492] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: No bootstrap required, opened a new log
I20260926 21:23:43.318956 22374 ts_tablet_manager.cc:1397] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260926 21:23:43.319694 22375 tablet_bootstrap.cc:492] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: No bootstrap required, opened a new log
I20260926 21:23:43.319994 22375 ts_tablet_manager.cc:1397] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260926 21:23:43.328502 22378 tablet_bootstrap.cc:492] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Bootstrap starting.
I20260926 21:23:43.328178 22374 raft_consensus.cc:348] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.328812 22374 raft_consensus.cc:374] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:43.328933 22374 raft_consensus.cc:729] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176d2d568df44be4b8bfe5ee05793052, State: Initialized, Role: FOLLOWER
I20260926 21:23:43.329800 22374 consensus_queue.cc:260] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.331861 22374 ts_tablet_manager.cc:1428] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 21:23:43.332037 22375 raft_consensus.cc:348] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.332523 22375 raft_consensus.cc:374] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:43.332664 22375 raft_consensus.cc:729] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bb5e784e894167804079ec5800aa57, State: Initialized, Role: FOLLOWER
I20260926 21:23:43.333436 22375 consensus_queue.cc:260] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.333714 22378 tablet_bootstrap.cc:654] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:43.335291 22378 log.cc:824] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:43.335507 22375 ts_tablet_manager.cc:1428] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260926 21:23:43.335820 22338 heartbeater.cc:503] Master 127.14.142.62:42663 was elected leader, sending a full tablet report...
I20260926 21:23:43.348258 22378 tablet_bootstrap.cc:492] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: No bootstrap required, opened a new log
I20260926 21:23:43.348644 22378 ts_tablet_manager.cc:1397] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.006s
I20260926 21:23:43.359884 22378 raft_consensus.cc:348] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.360291 22378 raft_consensus.cc:374] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:43.360363 22378 raft_consensus.cc:729] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16ca4cc2861428faad7658e6f5f7445, State: Initialized, Role: FOLLOWER
I20260926 21:23:43.361140 22378 consensus_queue.cc:260] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.362440 22197 heartbeater.cc:503] Master 127.14.142.62:42663 was elected leader, sending a full tablet report...
I20260926 21:23:43.363178 22378 ts_tablet_manager.cc:1428] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260926 21:23:43.393950 22340 tablet.cc:2369] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:23:43.471783 22199 tablet.cc:2369] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:43.497828 22379 raft_consensus.cc:482] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:43.497828 22382 raft_consensus.cc:482] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:43.498207 22382 raft_consensus.cc:504] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.498214 22379 raft_consensus.cc:504] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.502084 22379 leader_election.cc:288] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363), 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:43.506925 22382 leader_election.cc:288] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175), 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:43.522773 22386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.501773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:55959 (local address 127.14.142.3:37659)
0926 21:23:43.513364 (+ 11591us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.513371 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.513400 (+ 29us) server_negotiation.cc:407] Connection header received
0926 21:23:43.513483 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.513488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.513587 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.513691 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.514892 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.516411 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.519462 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.521011 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.521211 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.521289 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.522233 (+ 944us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.522387 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11417,"thread_start_us":11239,"threads_started":1}
I20260926 21:23:43.525123 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.501374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:33917 (local address 127.14.142.2:41363)
0926 21:23:43.502237 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.502246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.513594 (+ 11348us) server_negotiation.cc:407] Connection header received
0926 21:23:43.514145 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.514152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.514229 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.514300 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.515230 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.516403 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.522257 (+ 5854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.523713 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.523857 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.523930 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.524569 (+ 639us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.524708 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":684,"spinlock_wait_cycles":2432}
I20260926 21:23:43.525769 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.501051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:37659 (local address 127.14.142.2:55959)
0926 21:23:43.512415 (+ 11364us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.512446 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.512666 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.514199 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.514204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.514226 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.514685 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.514700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.516618 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.516623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.519258 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.519274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.521458 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.521465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.521674 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.521710 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.521809 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.521978 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11125,"mutex_wait_us":1,"thread_start_us":12839,"threads_started":1}
I20260926 21:23:43.526573 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.500880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41363 (local address 127.14.142.1:33917)
0926 21:23:43.513484 (+ 12604us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.513506 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.513962 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.514563 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.514570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.514613 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.515049 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.515064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.516637 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.516646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.522002 (+ 5356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.522018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.524145 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.524152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.524337 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.524373 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.525320 (+ 947us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.526141 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12445,"mutex_wait_us":1,"thread_start_us":16305,"threads_started":1}
I20260926 21:23:43.527092 22292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "d16ca4cc2861428faad7658e6f5f7445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bb5e784e894167804079ec5800aa57" is_pre_election: true
I20260926 21:23:43.527640 22292 raft_consensus.cc:2454] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d16ca4cc2861428faad7658e6f5f7445 in term 0.
I20260926 21:23:43.527746 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.517464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:56393 (local address 127.14.142.1:36175)
0926 21:23:43.517667 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.517673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.517692 (+ 19us) server_negotiation.cc:407] Connection header received
0926 21:23:43.517747 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.517753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.517830 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.517937 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.518679 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.520446 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.525491 (+ 5045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.526801 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.526975 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.527045 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.527395 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.527533 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":46}
I20260926 21:23:43.528040 22371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.501633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:45173 (local address 127.14.142.3:37659)
0926 21:23:43.501951 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.501962 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.517175 (+ 15213us) server_negotiation.cc:407] Connection header received
0926 21:23:43.517794 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.517799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.517875 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.517991 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:43.518933 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.521178 (+ 2245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.524226 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.525531 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.525662 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.525768 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.527523 (+ 1755us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.527663 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 21:23:43.528400 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.502578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36175 (local address 127.14.142.2:56393)
0926 21:23:43.517236 (+ 14658us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.517256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.517386 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.517980 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.517985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.518011 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.518440 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.518455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.521415 (+ 2960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.521421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.525346 (+ 3925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.525360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.527016 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.527023 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.527216 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.527250 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.527877 (+ 627us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.528042 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14519,"mutex_wait_us":1491,"thread_start_us":14896,"threads_started":1}
I20260926 21:23:43.528816 22383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.500993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:37659 (local address 127.14.142.1:45173)
0926 21:23:43.517046 (+ 16053us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:43.517070 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:43.517620 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:43.518185 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:43.518189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:43.518222 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:43.518731 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.518744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.521559 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.521564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.524017 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:43.524032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.527147 (+ 3115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.527153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:43.527336 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.527378 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 21:23:43.528283 (+ 905us) client_negotiation.cc:240] Negotiation successful
0926 21:23:43.528449 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15875,"mutex_wait_us":56,"thread_start_us":12621,"threads_started":1}
I20260926 21:23:43.529814 22004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "d16ca4cc2861428faad7658e6f5f7445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176d2d568df44be4b8bfe5ee05793052" is_pre_election: true
I20260926 21:23:43.529902 22291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "176d2d568df44be4b8bfe5ee05793052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bb5e784e894167804079ec5800aa57" is_pre_election: true
I20260926 21:23:43.530222 22291 raft_consensus.cc:2454] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176d2d568df44be4b8bfe5ee05793052 in term 0.
I20260926 21:23:43.530226 22004 raft_consensus.cc:2454] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d16ca4cc2861428faad7658e6f5f7445 in term 0.
I20260926 21:23:43.530905 22084 leader_election.cc:302] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176d2d568df44be4b8bfe5ee05793052, d16ca4cc2861428faad7658e6f5f7445; no voters:
I20260926 21:23:43.530855 21939 leader_election.cc:302] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176d2d568df44be4b8bfe5ee05793052, 25bb5e784e894167804079ec5800aa57; no voters:
I20260926 21:23:43.531067 22151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "176d2d568df44be4b8bfe5ee05793052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16ca4cc2861428faad7658e6f5f7445" is_pre_election: true
I20260926 21:23:43.531337 22382 raft_consensus.cc:2781] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:43.531508 22379 raft_consensus.cc:2781] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:43.531519 22151 raft_consensus.cc:2454] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176d2d568df44be4b8bfe5ee05793052 in term 0.
I20260926 21:23:43.531706 22382 raft_consensus.cc:482] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:43.531702 22379 raft_consensus.cc:482] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:43.531816 22382 raft_consensus.cc:3037] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:43.531868 22379 raft_consensus.cc:3037] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:43.534607 22379 raft_consensus.cc:504] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.535015 22382 raft_consensus.cc:504] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.535290 22379 leader_election.cc:288] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [CANDIDATE]: Term 1 election: Requested vote from peers d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363), 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:43.535660 22382 leader_election.cc:288] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [CANDIDATE]: Term 1 election: Requested vote from peers 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175), 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659)
I20260926 21:23:43.535917 22292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "176d2d568df44be4b8bfe5ee05793052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bb5e784e894167804079ec5800aa57"
I20260926 21:23:43.535949 22151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "176d2d568df44be4b8bfe5ee05793052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16ca4cc2861428faad7658e6f5f7445"
I20260926 21:23:43.536186 22151 raft_consensus.cc:2379] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 176d2d568df44be4b8bfe5ee05793052 in current term 1: Already voted for candidate d16ca4cc2861428faad7658e6f5f7445 in this term.
I20260926 21:23:43.536140 22292 raft_consensus.cc:3037] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:43.536188 22291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "d16ca4cc2861428faad7658e6f5f7445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bb5e784e894167804079ec5800aa57"
I20260926 21:23:43.537428 22004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31cfbd46870847f08f235940db3f90c7" candidate_uuid: "d16ca4cc2861428faad7658e6f5f7445" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176d2d568df44be4b8bfe5ee05793052"
I20260926 21:23:43.537681 22004 raft_consensus.cc:2379] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d16ca4cc2861428faad7658e6f5f7445 in current term 1: Already voted for candidate 176d2d568df44be4b8bfe5ee05793052 in this term.
I20260926 21:23:43.538223 22084 leader_election.cc:302] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d16ca4cc2861428faad7658e6f5f7445; no voters: 176d2d568df44be4b8bfe5ee05793052, 25bb5e784e894167804079ec5800aa57
I20260926 21:23:43.538537 22382 raft_consensus.cc:2726] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260926 21:23:43.539575 22292 raft_consensus.cc:2454] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176d2d568df44be4b8bfe5ee05793052 in term 1.
I20260926 21:23:43.540057 21939 leader_election.cc:302] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 176d2d568df44be4b8bfe5ee05793052, 25bb5e784e894167804079ec5800aa57; no voters: d16ca4cc2861428faad7658e6f5f7445
I20260926 21:23:43.540395 22379 raft_consensus.cc:2781] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:43.540884 22379 raft_consensus.cc:686] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 1 LEADER]: Becoming Leader. State: Replica: 176d2d568df44be4b8bfe5ee05793052, State: Running, Role: LEADER
I20260926 21:23:43.541366 22379 consensus_queue.cc:237] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:43.546572 21079 catalog_manager.cc:5495] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1). New cstate: current_term: 1 leader_uuid: "176d2d568df44be4b8bfe5ee05793052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:43.572901 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.566580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42494 (local address 127.14.142.1:36175)
0926 21:23:43.566760 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.566765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.566782 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:23:43.566873 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.566876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.566952 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.567091 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:43.567646 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.568711 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.570225 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.570813 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.570925 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.571483 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:43.572082 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:43.572205 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.572375 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.572470 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 21:23:43.572829 22052 tablet.cc:2369] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:43.581241 22292 raft_consensus.cc:1261] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Refusing update from remote peer 176d2d568df44be4b8bfe5ee05793052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:43.581305 22151 raft_consensus.cc:1261] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Refusing update from remote peer 176d2d568df44be4b8bfe5ee05793052: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:43.582180 22388 consensus_queue.cc:1035] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:43.582537 22379 consensus_queue.cc:1035] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:43.611347 22393 mvcc.cc:205] Tried to move back new op lower bound from 7333715245378723840 to 7333715245228879872. Current Snapshot: MvccSnapshot[applied={T|T < 7333715245378723840}]
I20260926 21:23:43.627527 21078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:43.627831 21078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:43.636446 21078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1e509ff7dc4430eb1417ebf8ed92b46: Sending DeleteTablet for 3 replicas of tablet 31cfbd46870847f08f235940db3f90c7
I20260926 21:23:43.637603 21984 tablet_service.cc:1514] Processing DeleteTablet for tablet 31cfbd46870847f08f235940db3f90c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:43 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:42492
I20260926 21:23:43.637748 22131 tablet_service.cc:1514] Processing DeleteTablet for tablet 31cfbd46870847f08f235940db3f90c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:43 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41898
I20260926 21:23:43.637748 22272 tablet_service.cc:1514] Processing DeleteTablet for tablet 31cfbd46870847f08f235940db3f90c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:43 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55360
I20260926 21:23:43.668026 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21047
I20260926 21:23:43.669602 21109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:43.672535 22403 tablet_replica.cc:330] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: stopping tablet replica
I20260926 21:23:43.674856 22403 raft_consensus.cc:2229] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:43.675576 22403 raft_consensus.cc:2258] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:43.677328 22402 tablet_replica.cc:330] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: stopping tablet replica
I20260926 21:23:43.677932 22402 raft_consensus.cc:2229] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:43.679133 22401 tablet_replica.cc:330] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: stopping tablet replica
I20260926 21:23:43.681166 22401 raft_consensus.cc:2229] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:43.682981 22403 ts_tablet_manager.cc:1905] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:43.683555 22401 raft_consensus.cc:2258] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:43.684963 22402 raft_consensus.cc:2258] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:43.687036 22401 ts_tablet_manager.cc:1905] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:43.690673 22403 ts_tablet_manager.cc:1918] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:43.690934 22403 log.cc:1196] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/31cfbd46870847f08f235940db3f90c7
I20260926 21:23:43.691599 22403 ts_tablet_manager.cc:1939] T 31cfbd46870847f08f235940db3f90c7 P 25bb5e784e894167804079ec5800aa57: Deleting consensus metadata
I20260926 21:23:43.695659 22402 ts_tablet_manager.cc:1905] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:43.697499 22401 ts_tablet_manager.cc:1918] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:43.697715 22401 log.cc:1196] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/31cfbd46870847f08f235940db3f90c7
I20260926 21:23:43.698254 22401 ts_tablet_manager.cc:1939] T 31cfbd46870847f08f235940db3f90c7 P d16ca4cc2861428faad7658e6f5f7445: Deleting consensus metadata
I20260926 21:23:43.703285 22402 ts_tablet_manager.cc:1918] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:43.703467 22402 log.cc:1196] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/31cfbd46870847f08f235940db3f90c7
I20260926 21:23:43.704104 22402 ts_tablet_manager.cc:1939] T 31cfbd46870847f08f235940db3f90c7 P 176d2d568df44be4b8bfe5ee05793052: Deleting consensus metadata
I20260926 21:23:43.706360 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.644191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.14.142.61:33303)
0926 21:23:43.644493 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.644503 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.665528 (+ 21025us) server_negotiation.cc:407] Connection header received
0926 21:23:43.669685 (+ 4157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.669691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.669797 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.669883 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.679714 (+ 9831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.681295 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.683346 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.684036 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.684165 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.695876 (+ 11711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.695885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.695892 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.695940 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.699491 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.700283 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.700288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.700291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.700428 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.700895 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.700899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.700902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.701286 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.701555 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.705643 (+ 4088us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.705953 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 21:23:43.721810 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.666349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.14.142.60:42243)
0926 21:23:43.666548 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.666554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.702809 (+ 36255us) server_negotiation.cc:407] Connection header received
0926 21:23:43.703143 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.703150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.703244 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.703321 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.706623 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.707840 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.709743 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.710656 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.710990 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.714583 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.714590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.714595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.714646 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.718557 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.719386 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.719392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.719395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.719561 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.719950 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.719957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.719961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.720362 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.720625 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.721017 (+ 392us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.721270 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 21:23:43.820940 21065 catalog_manager.cc:4841] TS 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659): tablet 31cfbd46870847f08f235940db3f90c7 (table test-table [id=a355a067656b4d31a07e5a4774ee69e9]) successfully deleted
I20260926 21:23:43.821842 21063 catalog_manager.cc:4841] TS 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175): tablet 31cfbd46870847f08f235940db3f90c7 (table test-table [id=a355a067656b4d31a07e5a4774ee69e9]) successfully deleted
I20260926 21:23:43.822575 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21047
I20260926 21:23:43.873073 22353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.854751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.14.142.60:42243)
0926 21:23:43.854960 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.854964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.856966 (+ 2002us) server_negotiation.cc:407] Connection header received
0926 21:23:43.857119 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.857125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.857249 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.857360 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.859605 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.860708 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.862331 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.862788 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.862938 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.866544 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.866550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.866554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.866587 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.869836 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.870719 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.870724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.870727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.870882 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.871327 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.871333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.871339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.871806 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.872060 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.872461 (+ 401us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.872737 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 21:23:43.874500 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:43.852284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.14.142.61:33303)
0926 21:23:43.852483 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:43.852490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:43.853786 (+ 1296us) server_negotiation.cc:407] Connection header received
0926 21:23:43.854149 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:43.854158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:43.854266 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:43.854367 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:43.856736 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.858024 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.860785 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:43.861455 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:43.861583 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:43.866503 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:43.866510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:43.866515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:43.866567 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:43.870410 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.871174 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.871179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.871183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.871332 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:43.871743 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:43.871748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:43.871752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:43.872145 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:43.872424 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:43.873806 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0926 21:23:43.874068 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:23:44.617063 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:44.616231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:40641)
0926 21:23:44.616770 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:44.616879 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":272,"threads_started":1}
W20260926 21:23:44.617553 22338 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:42663 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:44.617632 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:44.616800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:58699)
0926 21:23:44.617286 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:44.617429 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":267,"threads_started":1}
W20260926 21:23:44.618026 22050 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:42663 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:44.620626 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:44.619847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:53347)
0926 21:23:44.620274 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:44.620400 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":202,"threads_started":1}
W20260926 21:23:44.621080 22197 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:42663 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:45.135713 22415 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1e509ff7dc4430eb1417ebf8ed92b46)
I20260926 21:23:45.136065 22415 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } }
I20260926 21:23:45.136430 22416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b1e509ff7dc4430eb1417ebf8ed92b46)
I20260926 21:23:45.136745 22416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } }
I20260926 21:23:45.137876 22415 leader_election.cc:288] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663), 5071e9a0f47249dc89c78bbb6724a0da (127.14.142.61:33303)
I20260926 21:23:45.139408 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.137987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54114)
0926 21:23:45.138699 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.139012 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":385,"threads_started":1}
W20260926 21:23:45.139843 21855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:45.141595 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.138149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54122)
0926 21:23:45.141010 (+ 2861us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.141239 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2712,"thread_start_us":2546,"threads_started":1}
I20260926 21:23:45.142161 22416 leader_election.cc:288] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663), ff4dca5cf61342328bfd100adbc94828 (127.14.142.60:42243)
I20260926 21:23:45.142707 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.142045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54126)
0926 21:23:45.142433 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.142533 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260926 21:23:45.142796 21666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
W20260926 21:23:45.143002 21855 leader_election.cc:334] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663): Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:45.147661 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.146908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54132)
0926 21:23:45.147151 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.147408 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 21:23:45.148386 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.138256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:33303 (local address 127.0.0.1:47550)
0926 21:23:45.139102 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.139127 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.139278 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.140308 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.140318 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.140362 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.140874 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.140887 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.143071 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.143084 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.145746 (+ 2662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.145767 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.147454 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.147460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.147647 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.147688 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.147809 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.147953 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":39,"thread_start_us":592,"threads_started":1}
W20260926 21:23:45.148674 21666 leader_election.cc:334] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663): Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:45.148872 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.138687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.14.142.61:33303)
0926 21:23:45.139925 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.139932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.139956 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:23:45.140040 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.140045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.140128 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.140252 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.141512 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.142838 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.146007 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.147265 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.147439 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.147609 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.147925 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.148115 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1100,"thread_start_us":853,"threads_started":1}
I20260926 21:23:45.149482 21690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff4dca5cf61342328bfd100adbc94828" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5071e9a0f47249dc89c78bbb6724a0da" is_pre_election: true
I20260926 21:23:45.149816 21690 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff4dca5cf61342328bfd100adbc94828 in term 1.
I20260926 21:23:45.150352 21854 leader_election.cc:302] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5071e9a0f47249dc89c78bbb6724a0da, ff4dca5cf61342328bfd100adbc94828; no voters: b1e509ff7dc4430eb1417ebf8ed92b46
I20260926 21:23:45.150806 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.141365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:42243 (local address 127.0.0.1:41844)
0926 21:23:45.141829 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.141855 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.141969 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.143446 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.143454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.143485 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.143787 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.143800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.145965 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.145973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.148326 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.148343 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.149982 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.149989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.150237 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.150272 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.150375 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.150549 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":1110,"threads_started":1}
I20260926 21:23:45.150816 22415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 21:23:45.150931 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.141811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.14.142.60:42243)
0926 21:23:45.143060 (+ 1249us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.143066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.143092 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:23:45.143152 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.143159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.143259 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.143950 (+ 691us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.144300 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.145816 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.148622 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.149778 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.149928 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.150046 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.150402 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.150513 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":1096,"thread_start_us":1424,"threads_started":1}
I20260926 21:23:45.150990 22415 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b1e509ff7dc4430eb1417ebf8ed92b46)
I20260926 21:23:45.151114 22415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 1 FOLLOWER]: Advancing to term 2
I20260926 21:23:45.151499 21881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5071e9a0f47249dc89c78bbb6724a0da" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff4dca5cf61342328bfd100adbc94828" is_pre_election: true
I20260926 21:23:45.153797 22415 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } }
I20260926 21:23:45.154377 21881 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5071e9a0f47249dc89c78bbb6724a0da in current term 2: Already voted for candidate ff4dca5cf61342328bfd100adbc94828 in this term.
I20260926 21:23:45.154615 22415 leader_election.cc:288] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [CANDIDATE]: Term 2 election: Requested vote from peers b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663), 5071e9a0f47249dc89c78bbb6724a0da (127.14.142.61:33303)
I20260926 21:23:45.155252 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.154716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54146)
0926 21:23:45.154947 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.155045 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260926 21:23:45.155233 21690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff4dca5cf61342328bfd100adbc94828" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5071e9a0f47249dc89c78bbb6724a0da"
I20260926 21:23:45.155521 21690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 1 FOLLOWER]: Advancing to term 2
I20260926 21:23:45.155618 21666 leader_election.cc:302] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5071e9a0f47249dc89c78bbb6724a0da; no voters: b1e509ff7dc4430eb1417ebf8ed92b46, ff4dca5cf61342328bfd100adbc94828
I20260926 21:23:45.156293 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.155812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54148)
0926 21:23:45.156092 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.156170 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
W20260926 21:23:45.156454 21855 leader_election.cc:334] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663): Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
I20260926 21:23:45.158179 21690 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff4dca5cf61342328bfd100adbc94828 in term 2.
I20260926 21:23:45.158419 22416 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 21:23:45.158699 21854 leader_election.cc:302] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5071e9a0f47249dc89c78bbb6724a0da, ff4dca5cf61342328bfd100adbc94828; no voters: b1e509ff7dc4430eb1417ebf8ed92b46
I20260926 21:23:45.159063 22415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 21:23:45.159638 22415 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [term 2 LEADER]: Becoming Leader. State: Replica: ff4dca5cf61342328bfd100adbc94828, State: Running, Role: LEADER
I20260926 21:23:45.160137 22415 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } }
I20260926 21:23:45.162367 22425 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff4dca5cf61342328bfd100adbc94828. Latest consensus state: current_term: 2 leader_uuid: "ff4dca5cf61342328bfd100adbc94828" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:45.162784 22425 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:45.163869 22427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:45.168200 22427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:45.169209 22427 catalog_manager.cc:1259] Loaded cluster ID: 1ccf6a148e6e4d4db595e57b7ebb5b99
I20260926 21:23:45.169277 22427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:45.170437 22427 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:45.171244 22427 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Loaded TSK: 0
I20260926 21:23:45.172147 22427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:23:45.225605 21870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:45.237124 21870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:45.242969 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.242575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54172)
0926 21:23:45.242739 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.242830 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260926 21:23:45.243170 21690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [term 2 FOLLOWER]: Refusing update from remote peer ff4dca5cf61342328bfd100adbc94828: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 21:23:45.244172 22415 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260926 21:23:45.244799 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.244178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54186)
0926 21:23:45.244370 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.244470 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 21:23:45.245112 21855 consensus_peers.cc:487] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 -> Peer b1e509ff7dc4430eb1417ebf8ed92b46 (127.14.142.62:42663): Couldn't send request to peer b1e509ff7dc4430eb1417ebf8ed92b46. Status: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 21:23:45.248263 22416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff4dca5cf61342328bfd100adbc94828. Latest consensus state: current_term: 2 leader_uuid: "ff4dca5cf61342328bfd100adbc94828" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:45.248631 22416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:23:45.250353 22428 mvcc.cc:205] Tried to move back new op lower bound from 7333715252187451392 to 7333715251860262912. Current Snapshot: MvccSnapshot[applied={T|T < 7333715252187451392}]
I20260926 21:23:45.252770 22415 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "ff4dca5cf61342328bfd100adbc94828" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:45.252857 22416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "ff4dca5cf61342328bfd100adbc94828" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "b1e509ff7dc4430eb1417ebf8ed92b46" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42663 } } peers { permanent_uuid: "5071e9a0f47249dc89c78bbb6724a0da" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 33303 } attrs { promote: false } } peers { permanent_uuid: "ff4dca5cf61342328bfd100adbc94828" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 42243 } attrs { promote: false } } }
I20260926 21:23:45.253152 22415 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:45.253219 22416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5071e9a0f47249dc89c78bbb6724a0da [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:23:45.258401 22429 mvcc.cc:205] Tried to move back new op lower bound from 7333715252187451392 to 7333715251860262912. Current Snapshot: MvccSnapshot[applied={T|T < 7333715252187451392 or (T in {7333715252187451392})}]
I20260926 21:23:45.273216 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.262124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36175 (local address 127.0.0.1:42502)
0926 21:23:45.262385 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.262406 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.262528 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.263221 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.263229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.263265 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.263548 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.263583 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.265599 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.265608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.269754 (+ 4146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.269768 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.271946 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.271959 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.272303 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.272389 (+ 86us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.272475 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.272709 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":1437}
I20260926 21:23:45.274384 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.263252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41363 (local address 127.0.0.1:41914)
0926 21:23:45.263386 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.263405 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.263515 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.264520 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.264524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.264577 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.264871 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.264883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.266371 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.266379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.268969 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.268989 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.272038 (+ 3049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.272047 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.272332 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.273614 (+ 1282us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.273719 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.273933 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":1210}
I20260926 21:23:45.274966 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.263717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.14.142.2:41363)
0926 21:23:45.264182 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.264187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.264205 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:23:45.264275 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.264280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.264359 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.265002 (+ 643us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.265164 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.266196 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.270017 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.271756 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.272432 (+ 676us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.272647 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.274022 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.274249 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":274,"threads_started":1}
I20260926 21:23:45.275642 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.262190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42502 (local address 127.14.142.1:36175)
0926 21:23:45.262769 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.262778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.262804 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:23:45.262897 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.262906 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.263028 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.263738 (+ 710us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.263945 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.265379 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.269930 (+ 4551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.271715 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.274794 (+ 3079us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.274943 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.275029 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.275241 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":361,"threads_started":1}
I20260926 21:23:45.276100 22131 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8fb021e5d94f6d895e36538407284f (DEFAULT_TABLE table=test-table [id=4bfe2840c86d416f8bca8a2f53f43b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:45.276867 22131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8fb021e5d94f6d895e36538407284f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:45.276859 21984 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8fb021e5d94f6d895e36538407284f (DEFAULT_TABLE table=test-table [id=4bfe2840c86d416f8bca8a2f53f43b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:45.277689 21984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8fb021e5d94f6d895e36538407284f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:45.296473 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.265684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:37659 (local address 127.0.0.1:55376)
0926 21:23:45.273113 (+ 7429us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.273133 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.273256 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.283473 (+ 10217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.283478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.283508 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.284007 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.284023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.285688 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.285693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.290238 (+ 4545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.290265 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.295101 (+ 4836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.295113 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.295416 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.295484 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.295621 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.295875 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7298,"thread_start_us":7135,"threads_started":1}
I20260926 21:23:45.297291 22438 tablet_bootstrap.cc:492] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Bootstrap starting.
I20260926 21:23:45.297360 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.266053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.14.142.3:37659)
0926 21:23:45.281900 (+ 15847us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.281919 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.281941 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:23:45.282020 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.282026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.283120 (+ 1094us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.283241 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.284214 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.285502 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.290534 (+ 5032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.294463 (+ 3929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.294679 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.294892 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.296867 (+ 1975us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.297032 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15697,"thread_start_us":16781,"threads_started":1}
I20260926 21:23:45.298264 22272 tablet_service.cc:1467] Processing CreateTablet for tablet 4b8fb021e5d94f6d895e36538407284f (DEFAULT_TABLE table=test-table [id=4bfe2840c86d416f8bca8a2f53f43b7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:45.298933 22272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b8fb021e5d94f6d895e36538407284f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:45.301165 22439 tablet_bootstrap.cc:492] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Bootstrap starting.
I20260926 21:23:45.307322 22438 tablet_bootstrap.cc:654] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:45.308306 22439 tablet_bootstrap.cc:654] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:45.313642 22440 tablet_bootstrap.cc:492] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Bootstrap starting.
I20260926 21:23:45.316494 22440 tablet_bootstrap.cc:654] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:45.318919 22438 tablet_bootstrap.cc:492] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: No bootstrap required, opened a new log
I20260926 21:23:45.319123 22438 ts_tablet_manager.cc:1397] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.000s
I20260926 21:23:45.319856 22439 tablet_bootstrap.cc:492] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: No bootstrap required, opened a new log
I20260926 21:23:45.320035 22439 ts_tablet_manager.cc:1397] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.007s
I20260926 21:23:45.319965 22438 raft_consensus.cc:348] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.320235 22438 raft_consensus.cc:374] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:45.320358 22438 raft_consensus.cc:729] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d16ca4cc2861428faad7658e6f5f7445, State: Initialized, Role: FOLLOWER
I20260926 21:23:45.320706 22438 consensus_queue.cc:260] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.321049 22439 raft_consensus.cc:348] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.321349 22439 raft_consensus.cc:374] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:45.321465 22439 raft_consensus.cc:729] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176d2d568df44be4b8bfe5ee05793052, State: Initialized, Role: FOLLOWER
I20260926 21:23:45.321740 22439 consensus_queue.cc:260] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.322582 22440 tablet_bootstrap.cc:492] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: No bootstrap required, opened a new log
I20260926 21:23:45.322837 22440 ts_tablet_manager.cc:1397] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260926 21:23:45.324059 22440 raft_consensus.cc:348] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.324505 22440 raft_consensus.cc:374] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:45.324712 22440 raft_consensus.cc:729] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bb5e784e894167804079ec5800aa57, State: Initialized, Role: FOLLOWER
I20260926 21:23:45.325160 22440 consensus_queue.cc:260] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.329614 22438 ts_tablet_manager.cc:1428] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260926 21:23:45.336068 22196 heartbeater.cc:503] Master 127.14.142.60:42243 was elected leader, sending a full tablet report...
I20260926 21:23:45.354388 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.342342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:53467)
0926 21:23:45.353190 (+ 10848us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.353999 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10631,"thread_start_us":10178,"threads_started":1}
I20260926 21:23:45.355965 22439 ts_tablet_manager.cc:1428] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Time spent starting tablet: real 0.036s user 0.008s sys 0.011s
I20260926 21:23:45.362865 22440 ts_tablet_manager.cc:1428] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Time spent starting tablet: real 0.040s user 0.021s sys 0.018s
I20260926 21:23:45.365108 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.363766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:44349)
0926 21:23:45.364659 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.364826 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":706,"threads_started":1}
I20260926 21:23:45.366997 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.366542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:38773)
0926 21:23:45.366753 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.366859 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260926 21:23:45.367748 22337 heartbeater.cc:503] Master 127.14.142.60:42243 was elected leader, sending a full tablet report...
I20260926 21:23:45.371454 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.356722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:45499)
0926 21:23:45.370904 (+ 14182us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.371135 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14007,"threads_started":1}
I20260926 21:23:45.372217 22049 heartbeater.cc:503] Master 127.14.142.60:42243 was elected leader, sending a full tablet report...
I20260926 21:23:45.374960 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.373201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:37651)
0926 21:23:45.373400 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.374812 (+ 1412us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 21:23:45.378760 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.355249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:48709)
0926 21:23:45.376828 (+ 21579us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.376933 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21457,"threads_started":1}
I20260926 21:23:45.496132 22446 raft_consensus.cc:482] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:45.496400 22446 raft_consensus.cc:504] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.498127 22446 leader_election.cc:288] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175), d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363)
I20260926 21:23:45.506806 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.498537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:37821 (local address 127.14.142.1:36175)
0926 21:23:45.498755 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.498763 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.498787 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:23:45.498892 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.498900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.499002 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.499118 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.500016 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.501329 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.504448 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.505708 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.505845 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.505931 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.506389 (+ 458us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.506528 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:23:45.506827 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.498336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36175 (local address 127.14.142.3:37821)
0926 21:23:45.498532 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.498555 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.498680 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.499333 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.499340 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.499407 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.499815 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.499832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.501498 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.501504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.504260 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.504277 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.506096 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.506100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.506224 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.506247 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.506361 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.506521 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":28}
I20260926 21:23:45.507848 22004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fb021e5d94f6d895e36538407284f" candidate_uuid: "25bb5e784e894167804079ec5800aa57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176d2d568df44be4b8bfe5ee05793052" is_pre_election: true
I20260926 21:23:45.508014 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.499040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:41895 (local address 127.14.142.2:41363)
0926 21:23:45.499292 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.499301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.499916 (+ 615us) server_negotiation.cc:407] Connection header received
0926 21:23:45.500132 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.500137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.500196 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.500301 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:45.500812 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.501719 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.505065 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.506692 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.506851 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.506956 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.507586 (+ 630us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.507729 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:23:45.508128 22004 raft_consensus.cc:2454] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25bb5e784e894167804079ec5800aa57 in term 0.
I20260926 21:23:45.508324 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.498671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:41363 (local address 127.14.142.3:41895)
0926 21:23:45.499811 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.499825 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:45.499968 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:45.500315 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:45.500319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:45.500341 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:45.500672 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.500681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.501887 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.501895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.504845 (+ 2950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:45.504865 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.507131 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.507140 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:45.507397 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.507446 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 21:23:45.507558 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 21:23:45.507735 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":71,"thread_start_us":873,"threads_started":1}
I20260926 21:23:45.508775 22225 leader_election.cc:302] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176d2d568df44be4b8bfe5ee05793052, 25bb5e784e894167804079ec5800aa57; no voters:
I20260926 21:23:45.509202 22446 raft_consensus.cc:2781] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:45.509322 22446 raft_consensus.cc:482] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:45.509416 22446 raft_consensus.cc:3037] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:45.509639 22151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fb021e5d94f6d895e36538407284f" candidate_uuid: "25bb5e784e894167804079ec5800aa57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16ca4cc2861428faad7658e6f5f7445" is_pre_election: true
I20260926 21:23:45.509907 22151 raft_consensus.cc:2454] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25bb5e784e894167804079ec5800aa57 in term 0.
I20260926 21:23:45.511770 22446 raft_consensus.cc:504] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.512377 22446 leader_election.cc:288] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [CANDIDATE]: Term 1 election: Requested vote from peers 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175), d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363)
I20260926 21:23:45.512960 22151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fb021e5d94f6d895e36538407284f" candidate_uuid: "25bb5e784e894167804079ec5800aa57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d16ca4cc2861428faad7658e6f5f7445"
I20260926 21:23:45.513007 22004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b8fb021e5d94f6d895e36538407284f" candidate_uuid: "25bb5e784e894167804079ec5800aa57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176d2d568df44be4b8bfe5ee05793052"
I20260926 21:23:45.513191 22151 raft_consensus.cc:3037] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:45.513265 22004 raft_consensus.cc:3037] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:45.515393 22151 raft_consensus.cc:2454] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25bb5e784e894167804079ec5800aa57 in term 1.
I20260926 21:23:45.515815 22225 leader_election.cc:302] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25bb5e784e894167804079ec5800aa57, d16ca4cc2861428faad7658e6f5f7445; no voters:
I20260926 21:23:45.516132 22446 raft_consensus.cc:2781] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:45.517004 22004 raft_consensus.cc:2454] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25bb5e784e894167804079ec5800aa57 in term 1.
I20260926 21:23:45.517700 22446 raft_consensus.cc:686] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 1 LEADER]: Becoming Leader. State: Replica: 25bb5e784e894167804079ec5800aa57, State: Running, Role: LEADER
I20260926 21:23:45.518144 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.517652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:53319)
0926 21:23:45.517889 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.517994 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 21:23:45.518218 22446 consensus_queue.cc:237] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } }
I20260926 21:23:45.523280 21871 catalog_manager.cc:5495] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25bb5e784e894167804079ec5800aa57 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "25bb5e784e894167804079ec5800aa57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25bb5e784e894167804079ec5800aa57" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 37659 } health_report { overall_health: HEALTHY } } }
I20260926 21:23:45.587718 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.582286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.14.142.3:37659)
0926 21:23:45.582536 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.582544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.582564 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:23:45.582840 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.582844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.582925 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.583109 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:45.583456 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.584165 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.585564 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.585984 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.586121 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.586566 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:45.587115 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:45.587269 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.587347 (+ 78us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.587435 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260926 21:23:45.591974 22151 raft_consensus.cc:1261] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Refusing update from remote peer 25bb5e784e894167804079ec5800aa57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:45.592211 22004 raft_consensus.cc:1261] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Refusing update from remote peer 25bb5e784e894167804079ec5800aa57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:45.592962 22452 consensus_queue.cc:1035] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d16ca4cc2861428faad7658e6f5f7445" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 41363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:45.593413 22446 consensus_queue.cc:1035] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "176d2d568df44be4b8bfe5ee05793052" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:45.596076 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.595511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:51417)
0926 21:23:45.595744 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.595871 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260926 21:23:45.597178 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.596391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:49649)
0926 21:23:45.596706 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.596803 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260926 21:23:45.605043 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.604293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:57067)
0926 21:23:45.604728 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.604841 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306}
I20260926 21:23:45.609020 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.608199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.1:50491)
0926 21:23:45.608727 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.608830 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393}
I20260926 21:23:45.612998 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.612181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.2:33153)
0926 21:23:45.612615 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.612739 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241}
I20260926 21:23:45.613142 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.610475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.14.142.3:33329)
0926 21:23:45.612767 (+ 2292us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.612919 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2118}
I20260926 21:23:45.624092 21871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:45.624387 21871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:45.631142 21871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff4dca5cf61342328bfd100adbc94828: Sending DeleteTablet for 3 replicas of tablet 4b8fb021e5d94f6d895e36538407284f
I20260926 21:23:45.632966 22131 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8fb021e5d94f6d895e36538407284f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:45 UTC) from {username='oryx', principal='oryx/127.14.142.60@KRBTEST.COM'} at 127.0.0.1:41914
I20260926 21:23:45.632967 21984 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8fb021e5d94f6d895e36538407284f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:45 UTC) from {username='oryx', principal='oryx/127.14.142.60@KRBTEST.COM'} at 127.0.0.1:42502
I20260926 21:23:45.632998 22272 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b8fb021e5d94f6d895e36538407284f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:45 UTC) from {username='oryx', principal='oryx/127.14.142.60@KRBTEST.COM'} at 127.0.0.1:55376
I20260926 21:23:45.634316 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.633629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54556)
0926 21:23:45.633948 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.634079 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260926 21:23:45.634550 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21858
I20260926 21:23:45.637521 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.636889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54560)
0926 21:23:45.637150 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:45.637276 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260926 21:23:45.646389 22465 tablet_replica.cc:330] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: stopping tablet replica
I20260926 21:23:45.646643 22465 raft_consensus.cc:2229] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:45.647396 22466 tablet_replica.cc:330] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: stopping tablet replica
I20260926 21:23:45.647606 22466 raft_consensus.cc:2229] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:45.648247 22466 raft_consensus.cc:2258] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:45.648753 22465 raft_consensus.cc:2258] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:45.649592 22464 tablet_replica.cc:330] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: stopping tablet replica
I20260926 21:23:45.649782 22464 raft_consensus.cc:2229] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:45.650063 22464 raft_consensus.cc:2258] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:45.650444 22465 ts_tablet_manager.cc:1905] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:45.650605 22466 ts_tablet_manager.cc:1905] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:45.651348 22464 ts_tablet_manager.cc:1905] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:45.658075 22466 ts_tablet_manager.cc:1918] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:45.658082 22465 ts_tablet_manager.cc:1918] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:45.658175 22464 ts_tablet_manager.cc:1918] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:45.658231 22465 log.cc:1196] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/4b8fb021e5d94f6d895e36538407284f
I20260926 21:23:45.658321 22464 log.cc:1196] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/4b8fb021e5d94f6d895e36538407284f
I20260926 21:23:45.658744 22465 ts_tablet_manager.cc:1939] T 4b8fb021e5d94f6d895e36538407284f P 176d2d568df44be4b8bfe5ee05793052: Deleting consensus metadata
I20260926 21:23:45.658819 22464 ts_tablet_manager.cc:1939] T 4b8fb021e5d94f6d895e36538407284f P d16ca4cc2861428faad7658e6f5f7445: Deleting consensus metadata
I20260926 21:23:45.659067 22466 log.cc:1196] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/4b8fb021e5d94f6d895e36538407284f
I20260926 21:23:45.659569 22466 ts_tablet_manager.cc:1939] T 4b8fb021e5d94f6d895e36538407284f P 25bb5e784e894167804079ec5800aa57: Deleting consensus metadata
I20260926 21:23:45.660172 21854 catalog_manager.cc:4841] TS d16ca4cc2861428faad7658e6f5f7445 (127.14.142.2:41363): tablet 4b8fb021e5d94f6d895e36538407284f (table test-table [id=4bfe2840c86d416f8bca8a2f53f43b7e]) successfully deleted
I20260926 21:23:45.660514 21854 catalog_manager.cc:4841] TS 176d2d568df44be4b8bfe5ee05793052 (127.14.142.1:36175): tablet 4b8fb021e5d94f6d895e36538407284f (table test-table [id=4bfe2840c86d416f8bca8a2f53f43b7e]) successfully deleted
I20260926 21:23:45.660983 21856 catalog_manager.cc:4841] TS 25bb5e784e894167804079ec5800aa57 (127.14.142.3:37659): tablet 4b8fb021e5d94f6d895e36538407284f (table test-table [id=4bfe2840c86d416f8bca8a2f53f43b7e]) successfully deleted
I20260926 21:23:45.665519 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.635348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.14.142.1:36175)
0926 21:23:45.635542 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.635548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.639227 (+ 3679us) server_negotiation.cc:407] Connection header received
0926 21:23:45.645642 (+ 6415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.645662 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.645780 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.645901 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:45.648127 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.649325 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.651171 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.651765 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.651868 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.657946 (+ 6078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:45.657970 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:45.657995 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:45.658068 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:45.662232 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:45.663135 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:45.663142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:45.663144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:45.663279 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:45.663748 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:45.663753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:45.663755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:45.664363 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:45.664709 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.665006 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.665182 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:23:45.666164 22044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:45.813879 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21858
I20260926 21:23:45.837306 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 22061
I20260926 21:23:45.861699 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.838100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.14.142.2:41363)
0926 21:23:45.838273 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.838277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.838299 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:23:45.838548 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.838553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.838653 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.838744 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:45.840688 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.841414 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.843871 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.844339 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.844454 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.853736 (+ 9282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:45.853753 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:45.853769 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:45.853817 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:45.857121 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:45.858090 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:45.858098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:45.858100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:45.858228 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:45.858965 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:45.858972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:45.858975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:45.860048 (+ 1073us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:45.860494 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.860977 (+ 483us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.861254 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 21:23:45.862644 22191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:45.961091 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 22061
I20260926 21:23:45.980643 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 22209
I20260926 21:23:45.999070 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:45.981563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.14.142.3:37659)
0926 21:23:45.981731 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:45.981736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:45.981752 (+ 16us) server_negotiation.cc:407] Connection header received
0926 21:23:45.981806 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:45.981814 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:45.981875 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:45.981967 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:45.984524 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.985637 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.987510 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:45.988011 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:45.988153 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:45.991799 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:45.991815 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:45.991831 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:45.991873 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:45.995657 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:45.996711 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:45.996718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:45.996720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:45.996893 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:45.997330 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:45.997334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:45.997335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:45.997746 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:45.998001 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:45.998460 (+ 459us) server_negotiation.cc:299] Negotiation successful
0926 21:23:45.998640 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 21:23:45.999895 22332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:46.023475 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:46.022940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54566)
0926 21:23:46.023197 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:46.023306 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260926 21:23:46.024670 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:46.024236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:42663 (local address 127.0.0.1:54576)
0926 21:23:46.024410 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:46.024483 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:42663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 21:23:46.131840 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 22209
I20260926 21:23:46.153527 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21542
I20260926 21:23:46.154717 21711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:46.275007 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21542
I20260926 21:23:46.296314 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 21640
I20260926 21:23:46.297408 21901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:46.422050 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 21640
2026-09-26T21:23:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10490 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:46 dist-test-slave-4d5t krb5kdc[22475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:46 dist-test-slave-4d5t krb5kdc[22475](info): set up 2 sockets
Sep 26 21:23:46 dist-test-slave-4d5t krb5kdc[22475](info): commencing operation
krb5kdc: starting...
W20260926 21:23:47.566684 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:47 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab.
Entry for principal oryx/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab.
I20260926 21:23:47.806674 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:34439
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab
--principal=oryx/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:34439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:48.143460 22488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:48.143834 22488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:48.144009 22488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:48.151336 22488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:48.151489 22488 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 21:23:48.151587 22488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:48.151710 22488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:48.151794 22488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:48.151885 22488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:48.161823 22488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:34439
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.62.keytab
--principal=oryx/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:34439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:48.163139 22488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:48.164453 22488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:48.176975 22493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:48.177042 22496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:48.177009 22494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:48.179100 22488 server_base.cc:1034] running on GCE node
I20260926 21:23:48.180284 22488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:48.182179 22488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:48.183648 22488 hybrid_clock.cc:648] HybridClock initialized: now 1790457828183483 us; error 130 us; skew 500 ppm
I20260926 21:23:48.184072 22488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:48 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457828, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:48.187706 22488 init.cc:377] Logged in from keytab as oryx/127.14.142.62@KRBTEST.COM (short username oryx)
I20260926 21:23:48.189522 22488 webserver.cc:460] Webserver started at http://127.14.142.62:44761/ using document root <none> and password file <none>
I20260926 21:23:48.190287 22488 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:48.190428 22488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:48.190805 22488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:48.194265 22488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "fd5a0229b00541919c276cf8421bde90"
format_stamp: "Formatted at 2026-09-26 21:23:48 on dist-test-slave-4d5t"
server_key: "7fbc8d56f9dd3b1ddc6144605c28a1b1"
server_key_iv: "cadbc1f3db3a36a88732c84d78011933"
server_key_version: "encryptionkey@0"
I20260926 21:23:48.195184 22488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "fd5a0229b00541919c276cf8421bde90"
format_stamp: "Formatted at 2026-09-26 21:23:48 on dist-test-slave-4d5t"
server_key: "7fbc8d56f9dd3b1ddc6144605c28a1b1"
server_key_iv: "cadbc1f3db3a36a88732c84d78011933"
server_key_version: "encryptionkey@0"
I20260926 21:23:48.200739 22488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:48.204687 22503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:48.206141 22488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:23:48.206360 22488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "fd5a0229b00541919c276cf8421bde90"
format_stamp: "Formatted at 2026-09-26 21:23:48 on dist-test-slave-4d5t"
server_key: "7fbc8d56f9dd3b1ddc6144605c28a1b1"
server_key_iv: "cadbc1f3db3a36a88732c84d78011933"
server_key_version: "encryptionkey@0"
I20260926 21:23:48.206596 22488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:48.231990 22488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:48.236932 22488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:48.237171 22488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:48.259764 22488 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:34439
I20260926 21:23:48.259785 22564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:34439 every 8 connection(s)
I20260926 21:23:48.261269 22488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:48.265231 22565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:48.269915 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 22488
I20260926 21:23:48.270162 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:48.270795 14904 external_mini_cluster.cc:1428] Setting key 5596a77cd3f71137f64b6e4a76028b9b
I20260926 21:23:48.276631 22565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Bootstrap starting.
I20260926 21:23:48.280371 22565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:48.281710 22565 log.cc:824] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:48.285615 22565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: No bootstrap required, opened a new log
Sep 26 21:23:48 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:48.294689 22565 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5a0229b00541919c276cf8421bde90" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34439 } }
I20260926 21:23:48.295312 22565 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:48.295501 22565 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd5a0229b00541919c276cf8421bde90, State: Initialized, Role: FOLLOWER
I20260926 21:23:48.296394 22565 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5a0229b00541919c276cf8421bde90" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34439 } }
I20260926 21:23:48.296720 22565 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:48.296887 22565 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:48.297111 22565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:48.297824 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:48.272896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.14.142.62:34439)
0926 21:23:48.274097 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:48.274113 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:48.274160 (+ 47us) server_negotiation.cc:407] Connection header received
0926 21:23:48.275923 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:48.275945 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:48.276355 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:48.276790 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:48.278576 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.279957 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.281868 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.283043 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.283299 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:48.289080 (+ 5781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:48.289092 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:48.289109 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:48.289173 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:48.292449 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:48.293614 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:48.293621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:48.293628 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:48.293811 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:48.294235 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:48.294239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:48.294241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:48.294814 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:48.295050 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:48.295834 (+ 784us) server_negotiation.cc:299] Negotiation successful
0926 21:23:48.296418 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":896,"thread_start_us":507,"threads_started":1}
I20260926 21:23:48.299871 22565 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5a0229b00541919c276cf8421bde90" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34439 } }
I20260926 21:23:48.300453 22565 leader_election.cc:302] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd5a0229b00541919c276cf8421bde90; no voters:
I20260926 21:23:48.301167 22565 leader_election.cc:288] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:48.301273 22570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:48.302030 22570 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [term 1 LEADER]: Becoming Leader. State: Replica: fd5a0229b00541919c276cf8421bde90, State: Running, Role: LEADER
I20260926 21:23:48.302595 22570 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5a0229b00541919c276cf8421bde90" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34439 } }
I20260926 21:23:48.303323 22565 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:48.305984 22572 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd5a0229b00541919c276cf8421bde90. Latest consensus state: current_term: 1 leader_uuid: "fd5a0229b00541919c276cf8421bde90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5a0229b00541919c276cf8421bde90" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34439 } } }
I20260926 21:23:48.306591 22572 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:48.307199 22571 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd5a0229b00541919c276cf8421bde90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd5a0229b00541919c276cf8421bde90" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34439 } } }
I20260926 21:23:48.309073 22571 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90 [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:48.313493 22578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:48.319243 22578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:48.329749 22578 catalog_manager.cc:1347] Generated new cluster ID: 22729a29d1a142648b39f4ad3910d9df
I20260926 21:23:48.329905 22578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:48.353605 22578 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:48.355067 22578 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:48.370843 22578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Generated new TSK 0
I20260926 21:23:48.371899 22578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab.
Entry for principal oryx/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab.
I20260926 21:23:48.429150 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:34439
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:48.807300 22591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:48.807597 22591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:48.807786 22591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:48.817272 22591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:48.817524 22591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:48.817730 22591 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:23:48.817860 22591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:48.829638 22591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:34439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:48.830952 22591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:48.832983 22591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:48.849959 22597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:48.850068 22596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:48.850096 22599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:48.851253 22591 server_base.cc:1034] running on GCE node
I20260926 21:23:48.852093 22591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:48.853282 22591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:48.854591 22591 hybrid_clock.cc:648] HybridClock initialized: now 1790457828854506 us; error 43 us; skew 500 ppm
I20260926 21:23:48.854936 22591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:48 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457828, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:48.858759 22591 init.cc:377] Logged in from keytab as oryx/127.14.142.1@KRBTEST.COM (short username oryx)
I20260926 21:23:48.860474 22591 webserver.cc:460] Webserver started at http://127.14.142.1:36199/ using document root <none> and password file <none>
I20260926 21:23:48.861245 22591 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:48.861382 22591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:48.861723 22591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:48.865118 22591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "766ff97b80a3425590e5c8166cd2e031"
format_stamp: "Formatted at 2026-09-26 21:23:48 on dist-test-slave-4d5t"
server_key: "8a1b935a7a5e64055410ed397566606c"
server_key_iv: "5cf06d67f722a518f50896603eb301e7"
server_key_version: "encryptionkey@0"
I20260926 21:23:48.866016 22591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "766ff97b80a3425590e5c8166cd2e031"
format_stamp: "Formatted at 2026-09-26 21:23:48 on dist-test-slave-4d5t"
server_key: "8a1b935a7a5e64055410ed397566606c"
server_key_iv: "5cf06d67f722a518f50896603eb301e7"
server_key_version: "encryptionkey@0"
I20260926 21:23:48.872366 22591 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260926 21:23:48.876355 22606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:48.877941 22591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:23:48.878139 22591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "766ff97b80a3425590e5c8166cd2e031"
format_stamp: "Formatted at 2026-09-26 21:23:48 on dist-test-slave-4d5t"
server_key: "8a1b935a7a5e64055410ed397566606c"
server_key_iv: "5cf06d67f722a518f50896603eb301e7"
server_key_version: "encryptionkey@0"
I20260926 21:23:48.878388 22591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:48.911986 22591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:48.917471 22591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:48.917781 22591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:48.920868 22591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:48.920990 22591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:48.921167 22591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:48.921286 22591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:23:48 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457828, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:48.952062 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:48.921575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:47831 (local address 127.14.142.62:34439)
0926 21:23:48.922104 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:48.922113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:48.923244 (+ 1131us) server_negotiation.cc:407] Connection header received
0926 21:23:48.926500 (+ 3256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:48.926511 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:48.926834 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:48.927010 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:48.929949 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.931335 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.933559 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.934258 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.934416 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:48.942456 (+ 8040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:48.942467 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:48.942474 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:48.942528 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:48.946056 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:48.947655 (+ 1599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:48.947664 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:48.947671 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:48.947949 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:48.948948 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:48.948956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:48.948962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:48.949659 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:48.949871 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:48.951254 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0926 21:23:48.951570 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":284,"threads_started":1}
I20260926 21:23:48.954336 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:48.922180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.14.142.1:47831)
0926 21:23:48.923044 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:48.923102 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:48.926005 (+ 2903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:48.927342 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:48.927359 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:48.928376 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:48.929687 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:48.929721 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.931509 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.931513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:48.933371 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:48.933391 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.934803 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.934820 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:48.935319 (+ 499us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:48.936625 (+ 1306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:48.936672 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:48.942000 (+ 5328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:48.946280 (+ 4280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:48.946297 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:48.946321 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:48.947362 (+ 1041us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:48.948142 (+ 780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:48.948151 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:48.948163 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:48.948756 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:48.949871 (+ 1115us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:48.949899 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:48.950630 (+ 731us) client_negotiation.cc:769] Sending connection context
0926 21:23:48.951081 (+ 451us) client_negotiation.cc:240] Negotiation successful
0926 21:23:48.951963 (+ 882us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":335,"threads_started":1}
I20260926 21:23:48.969544 22591 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:39475
I20260926 21:23:48.969551 22726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:39475 every 8 connection(s)
I20260926 21:23:48.971218 22591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:48.971728 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 22591
I20260926 21:23:48.971961 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:48.972787 14904 external_mini_cluster.cc:1428] Setting key a031b97050744e2f7e3ac7135f4c4a46
I20260926 21:23:48.975497 22727 heartbeater.cc:347] Connected to a master server at 127.14.142.62:34439
I20260926 21:23:48.977095 22727 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:48.977932 22727 heartbeater.cc:511] Master 127.14.142.62:34439 requested a full tablet report, sending...
I20260926 21:23:48.980904 22520 ts_manager.cc:194] Registered new tserver with Master: 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475)
I20260926 21:23:48.982286 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:48.962566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:56369 (local address 127.14.142.62:34439)
0926 21:23:48.962749 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:48.962757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:48.964334 (+ 1577us) server_negotiation.cc:407] Connection header received
0926 21:23:48.964582 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:48.964592 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:48.964712 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:48.964828 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:48.965789 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.967142 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.968818 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.969656 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.969908 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:48.972950 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:48.972962 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:48.972971 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:48.973030 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:48.979409 (+ 6379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:48.980107 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:48.980114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:48.980118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:48.980293 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:48.980712 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:48.980717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:48.980720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:48.981035 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:48.981206 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:48.981619 (+ 413us) server_negotiation.cc:299] Negotiation successful
0926 21:23:48.981903 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 21:23:48.982335 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:48.963265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.14.142.1:56369)
0926 21:23:48.964181 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:48.964207 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:48.964393 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:48.964891 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:48.964895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:48.965333 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:48.965631 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:48.965640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.967292 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.967295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:48.968623 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:48.968644 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:48.969932 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:48.969937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:48.970073 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:48.971306 (+ 1233us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:48.971335 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:48.972610 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:48.979556 (+ 6946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:48.979564 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:48.979569 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:48.979971 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:48.980408 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:48.980413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:48.980417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:48.980548 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:48.981155 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:48.981160 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:48.981344 (+ 184us) client_negotiation.cc:769] Sending connection context
0926 21:23:48.981489 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 21:23:48.981856 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":517,"threads_started":1}
I20260926 21:23:48.983412 22520 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:47831
WARNING: no policy specified for oryx/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab.
Entry for principal oryx/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab.
I20260926 21:23:49.030206 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:34439
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:49.385571 22731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:49.385907 22731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:49.386081 22731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:49.393517 22731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:49.393671 22731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:49.393822 22731 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:23:49.393971 22731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:49.404212 22731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:34439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:49.405630 22731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:49.406977 22731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:49.420707 22736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:49.420732 22737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:49.420776 22739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:49.421536 22731 server_base.cc:1034] running on GCE node
I20260926 21:23:49.422214 22731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:49.423195 22731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:49.424454 22731 hybrid_clock.cc:648] HybridClock initialized: now 1790457829424368 us; error 60 us; skew 500 ppm
I20260926 21:23:49.424820 22731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:49 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457829, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:49.428215 22731 init.cc:377] Logged in from keytab as oryx/127.14.142.2@KRBTEST.COM (short username oryx)
I20260926 21:23:49.429885 22731 webserver.cc:460] Webserver started at http://127.14.142.2:45815/ using document root <none> and password file <none>
I20260926 21:23:49.430651 22731 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:49.430790 22731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:49.431124 22731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:49.434259 22731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
format_stamp: "Formatted at 2026-09-26 21:23:49 on dist-test-slave-4d5t"
server_key: "2938213de5079e44176295d8fb9f43a7"
server_key_iv: "f8a7558759c44999b37c98c3ddb54bc6"
server_key_version: "encryptionkey@0"
I20260926 21:23:49.435068 22731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
format_stamp: "Formatted at 2026-09-26 21:23:49 on dist-test-slave-4d5t"
server_key: "2938213de5079e44176295d8fb9f43a7"
server_key_iv: "f8a7558759c44999b37c98c3ddb54bc6"
server_key_version: "encryptionkey@0"
I20260926 21:23:49.440717 22731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:49.445509 22746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:49.447403 22731 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260926 21:23:49.447628 22731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
format_stamp: "Formatted at 2026-09-26 21:23:49 on dist-test-slave-4d5t"
server_key: "2938213de5079e44176295d8fb9f43a7"
server_key_iv: "f8a7558759c44999b37c98c3ddb54bc6"
server_key_version: "encryptionkey@0"
I20260926 21:23:49.447901 22731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:49.472083 22731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:49.477115 22731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:49.477437 22731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:49.480680 22731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:49.480806 22731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:49.481009 22731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:49.481105 22731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:23:49 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457829, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:49.507663 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:49.481871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.14.142.2:56805)
0926 21:23:49.482832 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:49.482879 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:49.484834 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:49.485656 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:49.485665 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:49.486223 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:49.487113 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:49.487139 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.488721 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.488725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:49.491156 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:49.491171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.492089 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.492097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:49.492327 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:49.493177 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:49.493225 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:49.496774 (+ 3549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:49.501799 (+ 5025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:49.501816 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:49.501844 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:49.502457 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:49.502932 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:49.502937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:49.502941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:49.503320 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:49.504256 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:49.504271 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:49.504912 (+ 641us) client_negotiation.cc:769] Sending connection context
0926 21:23:49.505307 (+ 395us) client_negotiation.cc:240] Negotiation successful
0926 21:23:49.505977 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":421,"threads_started":1}
I20260926 21:23:49.509261 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:49.481267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:56805 (local address 127.14.142.62:34439)
0926 21:23:49.481519 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:49.481529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:49.483044 (+ 1515us) server_negotiation.cc:407] Connection header received
0926 21:23:49.485139 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:49.485148 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:49.485270 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:49.485402 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:49.487423 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.488522 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.491341 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.491827 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.491976 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:49.497145 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:49.497245 (+ 100us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:49.497252 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:49.497308 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:49.501583 (+ 4275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:49.502608 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:49.502615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:49.502618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:49.502782 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:49.503600 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:49.503608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:49.503611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:49.503974 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:49.504166 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:49.508514 (+ 4348us) server_negotiation.cc:299] Negotiation successful
0926 21:23:49.508831 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:23:49.532963 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:49.515370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:42339 (local address 127.14.142.62:34439)
0926 21:23:49.515751 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:49.515763 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:49.516344 (+ 581us) server_negotiation.cc:407] Connection header received
0926 21:23:49.516529 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:49.516540 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:49.516737 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:49.516940 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:49.518138 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.519424 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.522251 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.522807 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.522929 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:49.526267 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:49.526275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:49.526283 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:49.526330 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:49.529390 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:49.530339 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:49.530344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:49.530348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:49.530542 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:49.531097 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:49.531104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:49.531109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:49.531527 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:49.531652 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:49.532298 (+ 646us) server_negotiation.cc:299] Negotiation successful
0926 21:23:49.532472 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260926 21:23:49.533452 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:49.515399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.14.142.2:42339)
0926 21:23:49.516215 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:49.516232 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:49.516379 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:49.516998 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:49.517002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:49.517545 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:49.517927 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:49.517946 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.519639 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.519648 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:49.522009 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:49.522043 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:49.523025 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:49.523042 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:49.523336 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:49.524131 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:49.524167 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:49.525981 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:49.529547 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:49.529553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:49.529557 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:49.530188 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:49.530685 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:49.530699 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:49.530704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:49.530918 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:49.531658 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:49.531666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:49.531954 (+ 288us) client_negotiation.cc:769] Sending connection context
0926 21:23:49.532186 (+ 232us) client_negotiation.cc:240] Negotiation successful
0926 21:23:49.532694 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":439,"threads_started":1}
I20260926 21:23:49.539358 22731 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:38441
I20260926 21:23:49.539381 22865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:38441 every 8 connection(s)
I20260926 21:23:49.540699 22731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:49.543068 22866 heartbeater.cc:347] Connected to a master server at 127.14.142.62:34439
I20260926 21:23:49.543526 22866 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:49.544544 22866 heartbeater.cc:511] Master 127.14.142.62:34439 requested a full tablet report, sending...
I20260926 21:23:49.546387 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 22731
I20260926 21:23:49.546414 22519 ts_manager.cc:194] Registered new tserver with Master: 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:49.546597 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:49.547039 14904 external_mini_cluster.cc:1428] Setting key 03120b17cf2db46e3d48bff2d1b5698d
I20260926 21:23:49.547897 22519 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:56805
WARNING: no policy specified for oryx/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab.
Entry for principal oryx/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab.
I20260926 21:23:49.601356 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:34439
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:37283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:49.962618 22870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:49.962999 22870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:49.963189 22870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:49.971413 22870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:49.971596 22870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:49.971756 22870 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 21:23:49.971908 22870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:49.982472 22870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.14.142.62:34439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:49.984004 22870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:49.985458 22870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 21:23:49.987159 22727 heartbeater.cc:503] Master 127.14.142.62:34439 was elected leader, sending a full tablet report...
W20260926 21:23:50.000105 22875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:50.000131 22876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:50.000066 22878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:50.000768 22870 server_base.cc:1034] running on GCE node
I20260926 21:23:50.001444 22870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:50.002656 22870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:50.003937 22870 hybrid_clock.cc:648] HybridClock initialized: now 1790457830003872 us; error 67 us; skew 500 ppm
I20260926 21:23:50.004289 22870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:50 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457830, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:50.009238 22870 init.cc:377] Logged in from keytab as oryx/127.14.142.3@KRBTEST.COM (short username oryx)
I20260926 21:23:50.011791 22870 webserver.cc:460] Webserver started at http://127.14.142.3:43671/ using document root <none> and password file <none>
I20260926 21:23:50.012867 22870 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:50.013041 22870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:50.013492 22870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:50.017066 22870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "ce9fcec9c4ca456891c246e9ae5c875d"
format_stamp: "Formatted at 2026-09-26 21:23:50 on dist-test-slave-4d5t"
server_key: "75af19bffaaf16429ffb0d747ce5e063"
server_key_iv: "69106c61c428964c007f827d0d71abf3"
server_key_version: "encryptionkey@0"
I20260926 21:23:50.017911 22870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "ce9fcec9c4ca456891c246e9ae5c875d"
format_stamp: "Formatted at 2026-09-26 21:23:50 on dist-test-slave-4d5t"
server_key: "75af19bffaaf16429ffb0d747ce5e063"
server_key_iv: "69106c61c428964c007f827d0d71abf3"
server_key_version: "encryptionkey@0"
I20260926 21:23:50.024151 22870 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 21:23:50.029328 22885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:50.031050 22870 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:50.031212 22870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "ce9fcec9c4ca456891c246e9ae5c875d"
format_stamp: "Formatted at 2026-09-26 21:23:50 on dist-test-slave-4d5t"
server_key: "75af19bffaaf16429ffb0d747ce5e063"
server_key_iv: "69106c61c428964c007f827d0d71abf3"
server_key_version: "encryptionkey@0"
I20260926 21:23:50.031497 22870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:50.050947 22870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:50.055979 22870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:50.056325 22870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:50.059706 22870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:50.059800 22870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:50.060029 22870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:50.060097 22870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:23:50 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457830, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:50.085824 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:50.060541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:39529 (local address 127.14.142.62:34439)
0926 21:23:50.063039 (+ 2498us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:50.063047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:50.063085 (+ 38us) server_negotiation.cc:407] Connection header received
0926 21:23:50.064673 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:50.064679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:50.064787 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:50.064924 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:50.066904 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.068158 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.070849 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.071573 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.071756 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:50.077154 (+ 5398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:50.077160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:50.077163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:50.077203 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:50.080476 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:50.081631 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:50.081638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:50.081642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:50.081784 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:50.082390 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:50.082398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:50.082403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:50.082734 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:50.082908 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:50.084873 (+ 1965us) server_negotiation.cc:299] Negotiation successful
0926 21:23:50.085218 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2325,"thread_start_us":2233,"threads_started":1}
I20260926 21:23:50.087059 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:50.061240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.14.142.3:39529)
0926 21:23:50.062224 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:50.062278 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:50.064335 (+ 2057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:50.065186 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:50.065195 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:50.065880 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:50.066704 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:50.066723 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.068416 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.068424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:50.070638 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:50.070664 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.071990 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.072011 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:50.072368 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:50.073408 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:50.073449 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:50.076855 (+ 3406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:50.080670 (+ 3815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:50.080685 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:50.080710 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:50.081491 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:50.081910 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:50.081915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:50.081922 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:50.082201 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:50.082872 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:50.082886 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:50.084036 (+ 1150us) client_negotiation.cc:769] Sending connection context
0926 21:23:50.084710 (+ 674us) client_negotiation.cc:240] Negotiation successful
0926 21:23:50.085502 (+ 792us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":459,"threads_started":1}
I20260926 21:23:50.115517 22870 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:43545
I20260926 21:23:50.115559 23005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:43545 every 8 connection(s)
I20260926 21:23:50.116961 22870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:50.117226 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:50.094021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:35401 (local address 127.14.142.62:34439)
0926 21:23:50.094247 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:50.094256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:50.097514 (+ 3258us) server_negotiation.cc:407] Connection header received
0926 21:23:50.097740 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:50.097749 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:50.097865 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:50.097980 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:50.100008 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.101338 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.104658 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.105370 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.105492 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:50.109674 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:50.109686 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:50.109693 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:50.109741 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:50.114139 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:50.115056 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:50.115061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:50.115065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:50.115204 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:50.115606 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:50.115611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:50.115615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:50.115960 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:50.116140 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:50.116580 (+ 440us) server_negotiation.cc:299] Negotiation successful
0926 21:23:50.116830 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 21:23:50.117353 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:50.094114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.14.142.3:35401)
0926 21:23:50.097373 (+ 3259us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:50.097392 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:50.097590 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:50.098221 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:50.098227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:50.099156 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:50.099765 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:50.099785 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.101560 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.101567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:50.104345 (+ 2778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:50.104368 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.105866 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.105881 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:50.106114 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:50.107056 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:50.107088 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:50.109245 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:50.114341 (+ 5096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:50.114347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:50.114350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:50.114903 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:50.115326 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:50.115331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:50.115335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:50.115481 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:50.116092 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:50.116097 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:50.116290 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 21:23:50.116449 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 21:23:50.116850 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3124,"thread_start_us":2976,"threads_started":1}
I20260926 21:23:50.119230 23006 heartbeater.cc:347] Connected to a master server at 127.14.142.62:34439
I20260926 21:23:50.119472 23006 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:50.120321 23006 heartbeater.cc:511] Master 127.14.142.62:34439 requested a full tablet report, sending...
I20260926 21:23:50.120640 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 22870
I20260926 21:23:50.120831 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:50.121353 14904 external_mini_cluster.cc:1428] Setting key 5f853395d0853c68b5d1275e56cfca49
I20260926 21:23:50.122027 22520 ts_manager.cc:194] Registered new tserver with Master: ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545)
I20260926 21:23:50.123075 22520 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:39529
I20260926 21:23:50.124948 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:23:50 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 21:23:50 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:23:50 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:23:50.551641 22866 heartbeater.cc:503] Master 127.14.142.62:34439 was elected leader, sending a full tablet report...
I20260926 21:23:51.126574 23006 heartbeater.cc:503] Master 127.14.142.62:34439 was elected leader, sending a full tablet report...
W20260926 21:23:51.320284 23014 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.142s user 0.005s sys 0.011s
W20260926 21:23:51.321707 22894 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:50.170169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.14.142.62:34439)
0926 21:23:50.170393 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:50.170409 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:50.170764 (+ 355us) server_negotiation.cc:407] Connection header received
0926 21:23:50.170981 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:50.170988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:50.171106 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:50.171297 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:50.173093 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.174657 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.176748 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:50.177211 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:50.177375 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.321358 (+1143983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55792: BlockingRecv error: recv got EOF from 127.0.0.1:55792 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 21:23:51 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:51.347702 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.326245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.14.142.62:34439)
0926 21:23:51.326498 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.326508 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.326854 (+ 346us) server_negotiation.cc:407] Connection header received
0926 21:23:51.327074 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.327079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.327144 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.327254 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:51.328448 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.329199 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.330467 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.330927 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.331067 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.341508 (+ 10441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:51.341515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:51.341519 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:51.341568 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:51.344798 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:51.345599 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:51.345606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:51.345612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:51.345745 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:51.346157 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:51.346163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:51.346167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:51.346464 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:51.346766 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.347103 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.347359 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 21:23:51.352262 22520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:51.360337 22520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:51.400331 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.373676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38441 (local address 127.0.0.1:46364)
0926 21:23:51.386855 (+ 13179us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:51.386897 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:51.387105 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:51.392273 (+ 5168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:51.392284 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:51.392316 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:51.392806 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.392824 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.394786 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.394795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.397759 (+ 2964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.397774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.399272 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.399277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.399464 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.399530 (+ 66us) client_negotiation.cc:769] Sending connection context
0926 21:23:51.399689 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 21:23:51.399897 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12891,"mutex_wait_us":98,"thread_start_us":17744,"threads_started":1}
I20260926 21:23:51.401230 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.373802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.14.142.2:38441)
0926 21:23:51.390827 (+ 17025us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.390839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.390877 (+ 38us) server_negotiation.cc:407] Connection header received
0926 21:23:51.391014 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.391027 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.391374 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.393008 (+ 1634us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:51.393207 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.394648 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.397956 (+ 3308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.399072 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.399240 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.399469 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.400773 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.400943 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16807,"thread_start_us":17724,"threads_started":1}
I20260926 21:23:51.401465 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.374702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.14.142.3:43545)
0926 21:23:51.391892 (+ 17190us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.391901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.392315 (+ 414us) server_negotiation.cc:407] Connection header received
0926 21:23:51.392420 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.392428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.392711 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.392930 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:51.393826 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.395330 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.398233 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.400066 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.400221 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.400419 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.400989 (+ 570us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.401162 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16986,"thread_start_us":16890,"threads_started":1}
I20260926 21:23:51.401237 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.374650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39475 (local address 127.0.0.1:57528)
0926 21:23:51.392114 (+ 17464us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:51.392132 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:51.392257 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:51.393173 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:51.393179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:51.393205 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:51.393873 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.393887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.395700 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.395707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.397860 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.397879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.400266 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.400273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.400471 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.400516 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 21:23:51.400639 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 21:23:51.400789 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17303,"mutex_wait_us":93,"thread_start_us":16208,"threads_started":1}
I20260926 21:23:51.401533 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.374042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43545 (local address 127.0.0.1:41700)
0926 21:23:51.392024 (+ 17982us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:51.392044 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:51.392182 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:51.392925 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:51.392930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:51.392953 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:51.393583 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.393599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.395532 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.395538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.398073 (+ 2535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.398085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.400605 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.400615 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.400800 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.400832 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 21:23:51.400938 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 21:23:51.401076 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17845,"mutex_wait_us":40,"thread_start_us":12382,"threads_started":1}
I20260926 21:23:51.401940 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.375374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.14.142.1:39475)
0926 21:23:51.383800 (+ 8426us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.383815 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.392389 (+ 8574us) server_negotiation.cc:407] Connection header received
0926 21:23:51.392502 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.392508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.392764 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.392954 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:51.394075 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.395532 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.398086 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.399718 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.399872 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.400084 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.401381 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.401594 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8170,"thread_start_us":7989,"threads_started":1}
I20260926 21:23:51.405933 22936 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed7a4196cbe41b285a15a0f104dff13 (DEFAULT_TABLE table=test-table [id=01d66115c33c400781f1907b3a375c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:51.405969 22657 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed7a4196cbe41b285a15a0f104dff13 (DEFAULT_TABLE table=test-table [id=01d66115c33c400781f1907b3a375c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:51.406699 22796 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed7a4196cbe41b285a15a0f104dff13 (DEFAULT_TABLE table=test-table [id=01d66115c33c400781f1907b3a375c13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:51.407238 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed7a4196cbe41b285a15a0f104dff13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:51.407447 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed7a4196cbe41b285a15a0f104dff13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:51.408061 22796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed7a4196cbe41b285a15a0f104dff13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:51.420171 23031 tablet_bootstrap.cc:492] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Bootstrap starting.
I20260926 21:23:51.423615 23032 tablet_bootstrap.cc:492] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Bootstrap starting.
I20260926 21:23:51.423280 23033 tablet_bootstrap.cc:492] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Bootstrap starting.
I20260926 21:23:51.424191 23031 tablet_bootstrap.cc:654] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:51.425583 23031 log.cc:824] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:51.427065 23033 tablet_bootstrap.cc:654] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:51.427551 23032 tablet_bootstrap.cc:654] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:51.428592 23033 log.cc:824] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:51.428810 23032 log.cc:824] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Log is configured to *not* fsync() on all Append() calls
I20260926 21:23:51.429529 23031 tablet_bootstrap.cc:492] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: No bootstrap required, opened a new log
I20260926 21:23:51.429821 23031 ts_tablet_manager.cc:1397] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 21:23:51.431689 23033 tablet_bootstrap.cc:492] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: No bootstrap required, opened a new log
I20260926 21:23:51.431704 23032 tablet_bootstrap.cc:492] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: No bootstrap required, opened a new log
I20260926 21:23:51.431962 23033 ts_tablet_manager.cc:1397] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260926 21:23:51.431977 23032 ts_tablet_manager.cc:1397] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260926 21:23:51.438230 23031 raft_consensus.cc:348] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.438766 23031 raft_consensus.cc:374] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:51.438892 23031 raft_consensus.cc:729] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Initialized, Role: FOLLOWER
I20260926 21:23:51.439637 23031 consensus_queue.cc:260] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.441471 23031 ts_tablet_manager.cc:1428] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260926 21:23:51.444047 23032 raft_consensus.cc:348] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.444047 23033 raft_consensus.cc:348] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.444545 23032 raft_consensus.cc:374] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:51.444545 23033 raft_consensus.cc:374] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:51.444701 23032 raft_consensus.cc:729] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766ff97b80a3425590e5c8166cd2e031, State: Initialized, Role: FOLLOWER
I20260926 21:23:51.444701 23033 raft_consensus.cc:729] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f356cfe3754636bb2ce9424b1b5ef1, State: Initialized, Role: FOLLOWER
I20260926 21:23:51.445411 23032 consensus_queue.cc:260] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.445410 23033 consensus_queue.cc:260] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.447624 23033 ts_tablet_manager.cc:1428] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260926 21:23:51.447732 23032 ts_tablet_manager.cc:1428] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
W20260926 21:23:51.477897 22728 tablet.cc:2369] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:51.523263 23037 raft_consensus.cc:482] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:51.523663 23037 raft_consensus.cc:504] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.525491 23037 leader_election.cc:288] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441), 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475)
I20260926 21:23:51.533551 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.525920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:42481 (local address 127.14.142.1:39475)
0926 21:23:51.526119 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.526123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.526753 (+ 630us) server_negotiation.cc:407] Connection header received
0926 21:23:51.527004 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.527008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.527067 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.527166 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:51.527827 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.528951 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.531084 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.532244 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.532381 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.532461 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.533096 (+ 635us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.533286 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 21:23:51.533560 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.525934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39475 (local address 127.14.142.3:42481)
0926 21:23:51.526633 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:51.526649 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:51.526834 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:51.527184 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:51.527189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:51.527213 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:51.527620 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.527633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.529082 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.529088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.530840 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.530855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.532701 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.532710 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.532888 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.532933 (+ 45us) client_negotiation.cc:769] Sending connection context
0926 21:23:51.533020 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 21:23:51.533121 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":68,"thread_start_us":375,"threads_started":1}
I20260926 21:23:51.533701 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.526114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:59359 (local address 127.14.142.2:38441)
0926 21:23:51.526345 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.526353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.527460 (+ 1107us) server_negotiation.cc:407] Connection header received
0926 21:23:51.527620 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.527626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.527713 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.527803 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:51.528498 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.529372 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.531398 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.532649 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.532809 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.532879 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.533310 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.533443 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 21:23:51.533990 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.526584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38441 (local address 127.14.142.3:59359)
0926 21:23:51.527330 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:51.527354 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:51.527481 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:51.527874 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:51.527880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:51.527915 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:51.528319 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.528331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.529528 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.529532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.531228 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:51.531241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.532895 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.532901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:51.533104 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.533141 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 21:23:51.533276 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 21:23:51.533436 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":20,"thread_start_us":391,"threads_started":1}
I20260926 21:23:51.534991 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a4196cbe41b285a15a0f104dff13" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031" is_pre_election: true
I20260926 21:23:51.534989 22816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a4196cbe41b285a15a0f104dff13" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" is_pre_election: true
I20260926 21:23:51.535567 22677 raft_consensus.cc:2454] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:51.535573 22816 raft_consensus.cc:2454] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:51.536324 22888 leader_election.cc:302] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21f356cfe3754636bb2ce9424b1b5ef1, 766ff97b80a3425590e5c8166cd2e031, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:51.536821 23037 raft_consensus.cc:2781] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:51.536999 23037 raft_consensus.cc:482] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:51.537113 23037 raft_consensus.cc:3037] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:51.539613 23037 raft_consensus.cc:504] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.540349 23037 leader_election.cc:288] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Requested vote from peers 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441), 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475)
I20260926 21:23:51.541018 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a4196cbe41b285a15a0f104dff13" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031"
I20260926 21:23:51.541040 22816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a4196cbe41b285a15a0f104dff13" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
I20260926 21:23:51.541260 22816 raft_consensus.cc:3037] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:51.541270 22677 raft_consensus.cc:3037] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Advancing to term 1
W20260926 21:23:51.544024 22867 tablet.cc:2369] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:51.544577 22677 raft_consensus.cc:2454] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:51.544708 22816 raft_consensus.cc:2454] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:51.545104 22888 leader_election.cc:302] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766ff97b80a3425590e5c8166cd2e031, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:51.545444 23037 raft_consensus.cc:2781] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:51.545928 23037 raft_consensus.cc:686] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Becoming Leader. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Running, Role: LEADER
I20260926 21:23:51.546492 23037 consensus_queue.cc:237] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } }
I20260926 21:23:51.552767 22520 catalog_manager.cc:5495] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3). New cstate: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:51.593046 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:51.587620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.14.142.3:43545)
0926 21:23:51.587806 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:51.587811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:51.587829 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:23:51.588023 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:51.588026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:51.588103 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:51.588229 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:51.588533 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.589265 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.590711 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:51.591137 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:51.591235 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:51.591602 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:51.592143 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:51.592257 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:51.592540 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 21:23:51.592764 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 21:23:51.601948 22677 raft_consensus.cc:1261] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Refusing update from remote peer ce9fcec9c4ca456891c246e9ae5c875d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:51.601948 22816 raft_consensus.cc:1261] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Refusing update from remote peer ce9fcec9c4ca456891c246e9ae5c875d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:51.603184 23037 consensus_queue.cc:1035] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:51.603546 23042 consensus_queue.cc:1035] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:51.635000 22519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:51.635248 22519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:51.639552 22519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Sending DeleteTablet for 3 replicas of tablet 1ed7a4196cbe41b285a15a0f104dff13
I20260926 21:23:51.640985 22796 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed7a4196cbe41b285a15a0f104dff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:51 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:46364
I20260926 21:23:51.641367 22657 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed7a4196cbe41b285a15a0f104dff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:51 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:57528
I20260926 21:23:51.641816 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed7a4196cbe41b285a15a0f104dff13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:51 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41700
I20260926 21:23:51.642302 23056 tablet_replica.cc:330] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: stopping tablet replica
I20260926 21:23:51.642812 23056 raft_consensus.cc:2229] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:51.643416 23056 raft_consensus.cc:2258] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:51.643254 22519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:51.644193 22519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:51.645810 23056 ts_tablet_manager.cc:1905] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:51.653501 23056 ts_tablet_manager.cc:1918] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:51.653391 22935 tablet_service.cc:1467] Processing CreateTablet for tablet ff173e671f3f4b8e845e1aab80be7a54 (DEFAULT_TABLE table=test-table [id=2232033837f24c44afb1c61c12f3e47b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:51.653846 23056 log.cc:1196] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/1ed7a4196cbe41b285a15a0f104dff13
I20260926 21:23:51.654111 22935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff173e671f3f4b8e845e1aab80be7a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:51.654639 23056 ts_tablet_manager.cc:1939] T 1ed7a4196cbe41b285a15a0f104dff13 P 766ff97b80a3425590e5c8166cd2e031: Deleting consensus metadata
I20260926 21:23:51.656306 22657 tablet_service.cc:1467] Processing CreateTablet for tablet ff173e671f3f4b8e845e1aab80be7a54 (DEFAULT_TABLE table=test-table [id=2232033837f24c44afb1c61c12f3e47b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:51.656790 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff173e671f3f4b8e845e1aab80be7a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:51.656777 22506 catalog_manager.cc:4841] TS 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475): tablet 1ed7a4196cbe41b285a15a0f104dff13 (table test-table [id=01d66115c33c400781f1907b3a375c13]) successfully deleted
I20260926 21:23:51.661768 22795 tablet_service.cc:1467] Processing CreateTablet for tablet ff173e671f3f4b8e845e1aab80be7a54 (DEFAULT_TABLE table=test-table [id=2232033837f24c44afb1c61c12f3e47b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:51.662328 22795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff173e671f3f4b8e845e1aab80be7a54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:51.666806 23031 tablet_bootstrap.cc:492] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Bootstrap starting.
I20260926 21:23:51.668018 23057 tablet_replica.cc:330] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: stopping tablet replica
I20260926 21:23:51.668503 23057 raft_consensus.cc:2229] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:51.668821 23031 tablet_bootstrap.cc:654] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:51.669384 23057 raft_consensus.cc:2258] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:51.670125 23055 tablet_replica.cc:330] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: stopping tablet replica
I20260926 21:23:51.670578 23055 raft_consensus.cc:2229] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:51.671284 23055 raft_consensus.cc:2258] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:51.672466 23057 ts_tablet_manager.cc:1905] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:51.673606 23055 ts_tablet_manager.cc:1905] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:51.673979 23033 tablet_bootstrap.cc:492] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Bootstrap starting.
I20260926 21:23:51.675261 23032 tablet_bootstrap.cc:492] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Bootstrap starting.
I20260926 21:23:51.675585 23033 tablet_bootstrap.cc:654] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:51.676991 23032 tablet_bootstrap.cc:654] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:51.679912 23033 tablet_bootstrap.cc:492] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: No bootstrap required, opened a new log
I20260926 21:23:51.680207 23033 ts_tablet_manager.cc:1397] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 21:23:51.682708 23031 tablet_bootstrap.cc:492] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: No bootstrap required, opened a new log
I20260926 21:23:51.682961 23031 ts_tablet_manager.cc:1397] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.003s
I20260926 21:23:51.682696 23033 raft_consensus.cc:348] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.683247 23033 raft_consensus.cc:374] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:51.683410 23033 raft_consensus.cc:729] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f356cfe3754636bb2ce9424b1b5ef1, State: Initialized, Role: FOLLOWER
I20260926 21:23:51.683843 23033 consensus_queue.cc:260] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.684085 23031 raft_consensus.cc:348] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.684342 23031 raft_consensus.cc:374] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:51.684479 23031 raft_consensus.cc:729] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Initialized, Role: FOLLOWER
I20260926 21:23:51.684826 23031 consensus_queue.cc:260] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.685456 23031 ts_tablet_manager.cc:1428] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 21:23:51.687041 23033 ts_tablet_manager.cc:1428] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260926 21:23:51.690111 23055 ts_tablet_manager.cc:1918] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:51.691025 23055 log.cc:1196] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/1ed7a4196cbe41b285a15a0f104dff13
I20260926 21:23:51.691107 23057 ts_tablet_manager.cc:1918] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:51.691318 23057 log.cc:1196] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/1ed7a4196cbe41b285a15a0f104dff13
I20260926 21:23:51.691658 23055 ts_tablet_manager.cc:1939] T 1ed7a4196cbe41b285a15a0f104dff13 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting consensus metadata
I20260926 21:23:51.691854 23057 ts_tablet_manager.cc:1939] T 1ed7a4196cbe41b285a15a0f104dff13 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting consensus metadata
I20260926 21:23:51.693621 22504 catalog_manager.cc:4841] TS 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441): tablet 1ed7a4196cbe41b285a15a0f104dff13 (table test-table [id=01d66115c33c400781f1907b3a375c13]) successfully deleted
I20260926 21:23:51.693650 22507 catalog_manager.cc:4841] TS ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545): tablet 1ed7a4196cbe41b285a15a0f104dff13 (table test-table [id=01d66115c33c400781f1907b3a375c13]) successfully deleted
I20260926 21:23:51.694482 23032 tablet_bootstrap.cc:492] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: No bootstrap required, opened a new log
I20260926 21:23:51.694666 23032 ts_tablet_manager.cc:1397] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.004s
I20260926 21:23:51.695592 23032 raft_consensus.cc:348] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.695844 23032 raft_consensus.cc:374] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:51.696053 23032 raft_consensus.cc:729] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766ff97b80a3425590e5c8166cd2e031, State: Initialized, Role: FOLLOWER
I20260926 21:23:51.696383 23032 consensus_queue.cc:260] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.697031 23032 ts_tablet_manager.cc:1428] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260926 21:23:51.872129 23007 tablet.cc:2369] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:23:51.911948 23037 raft_consensus.cc:482] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:51.912235 23037 raft_consensus.cc:504] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.913050 23037 leader_election.cc:288] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:51.913668 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff173e671f3f4b8e845e1aab80be7a54" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031" is_pre_election: true
I20260926 21:23:51.913731 22816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff173e671f3f4b8e845e1aab80be7a54" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" is_pre_election: true
I20260926 21:23:51.913954 22677 raft_consensus.cc:2454] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:51.913964 22816 raft_consensus.cc:2454] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:51.914407 22886 leader_election.cc:302] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21f356cfe3754636bb2ce9424b1b5ef1, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:51.914757 23037 raft_consensus.cc:2781] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:51.914912 23037 raft_consensus.cc:482] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:51.915024 23037 raft_consensus.cc:3037] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:51.918671 23037 raft_consensus.cc:504] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.919632 23037 leader_election.cc:288] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Requested vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:51.920102 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff173e671f3f4b8e845e1aab80be7a54" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031"
I20260926 21:23:51.920348 22677 raft_consensus.cc:3037] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:51.920684 22816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff173e671f3f4b8e845e1aab80be7a54" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
I20260926 21:23:51.920964 22816 raft_consensus.cc:3037] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:51.922765 22677 raft_consensus.cc:2454] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:51.923947 22888 leader_election.cc:302] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766ff97b80a3425590e5c8166cd2e031, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:51.924402 22816 raft_consensus.cc:2454] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:51.924496 23037 raft_consensus.cc:2781] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:51.924778 23037 raft_consensus.cc:686] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Becoming Leader. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Running, Role: LEADER
I20260926 21:23:51.925156 23037 consensus_queue.cc:237] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:51.929044 22518 catalog_manager.cc:5495] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3). New cstate: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } health_report { overall_health: HEALTHY } } }
Sep 26 21:23:52 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457828, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.62@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:52.025650 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.007192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34439 (local address 127.0.0.1:55808)
0926 21:23:52.008426 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:52.008449 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:52.008657 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:52.009082 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:52.009086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:52.010008 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:52.010338 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.010349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.012075 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.012082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.013781 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.013797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.014900 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.014906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.015047 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.015874 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:52.015900 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:52.018562 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:52.023058 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:52.023071 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:52.023078 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:52.023527 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:52.023954 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:52.023957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:52.023960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:52.024154 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:52.024843 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:52.024850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:52.025000 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 21:23:52.025133 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 21:23:52.025366 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":817,"threads_started":1}
I20260926 21:23:52.025995 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.007274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.14.142.62:34439)
0926 21:23:52.008234 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.008241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.008574 (+ 333us) server_negotiation.cc:407] Connection header received
0926 21:23:52.008795 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.008803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.008898 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.009047 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:52.010544 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.011785 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.013997 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.014690 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.014838 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.018955 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:52.018964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:52.018970 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:52.019017 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:52.022868 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:52.023686 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:52.023692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:52.023697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:52.023841 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:52.024275 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:52.024280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:52.024284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:52.024707 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:52.024869 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.025287 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.025544 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":390,"threads_started":1}
I20260926 21:23:52.032747 22518 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55808:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 21:23:52.034147 22518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:52.042418 22657 tablet_service.cc:1467] Processing CreateTablet for tablet ba3f9ea40edb4238aabfb1aa16a80a06 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4833628ecabc44af97a72974c0be3d19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:23:52.042979 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3f9ea40edb4238aabfb1aa16a80a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:52.042886 22795 tablet_service.cc:1467] Processing CreateTablet for tablet ba3f9ea40edb4238aabfb1aa16a80a06 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4833628ecabc44af97a72974c0be3d19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:23:52.043434 22795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3f9ea40edb4238aabfb1aa16a80a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:52.048339 23032 tablet_bootstrap.cc:492] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: Bootstrap starting.
I20260926 21:23:52.049899 23032 tablet_bootstrap.cc:654] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:52.051152 22936 tablet_service.cc:1467] Processing CreateTablet for tablet ba3f9ea40edb4238aabfb1aa16a80a06 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4833628ecabc44af97a72974c0be3d19]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 21:23:52.051800 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3f9ea40edb4238aabfb1aa16a80a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:52.060168 23031 tablet_bootstrap.cc:492] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: Bootstrap starting.
I20260926 21:23:52.061326 23033 tablet_bootstrap.cc:492] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: Bootstrap starting.
I20260926 21:23:52.063016 23032 tablet_bootstrap.cc:492] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: No bootstrap required, opened a new log
I20260926 21:23:52.063184 23033 tablet_bootstrap.cc:654] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:52.063218 23032 ts_tablet_manager.cc:1397] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260926 21:23:52.063387 23031 tablet_bootstrap.cc:654] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:52.064011 23032 raft_consensus.cc:348] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.064254 23032 raft_consensus.cc:374] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:52.064323 23032 raft_consensus.cc:729] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766ff97b80a3425590e5c8166cd2e031, State: Initialized, Role: FOLLOWER
I20260926 21:23:52.064646 23032 consensus_queue.cc:260] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.065117 23038 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.065445 23038 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.066915 23032 ts_tablet_manager.cc:1428] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 21:23:52.073972 23033 tablet_bootstrap.cc:492] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: No bootstrap required, opened a new log
I20260926 21:23:52.074177 23033 ts_tablet_manager.cc:1397] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.008s
I20260926 21:23:52.074826 23031 tablet_bootstrap.cc:492] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: No bootstrap required, opened a new log
I20260926 21:23:52.074810 23033 raft_consensus.cc:348] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.075021 23031 ts_tablet_manager.cc:1397] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260926 21:23:52.075099 23033 raft_consensus.cc:374] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:52.075212 23033 raft_consensus.cc:729] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f356cfe3754636bb2ce9424b1b5ef1, State: Initialized, Role: FOLLOWER
I20260926 21:23:52.075472 23033 consensus_queue.cc:260] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.075928 23039 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.075989 23031 raft_consensus.cc:348] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.076270 23039 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.076268 23031 raft_consensus.cc:374] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:52.076344 23031 raft_consensus.cc:729] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Initialized, Role: FOLLOWER
I20260926 21:23:52.076678 23031 consensus_queue.cc:260] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.077337 23031 ts_tablet_manager.cc:1428] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 21:23:52.077415 23037 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.077811 23037 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.078686 23033 ts_tablet_manager.cc:1428] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260926 21:23:52.410805 23037 raft_consensus.cc:482] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:52.411103 23037 raft_consensus.cc:504] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.411789 23037 leader_election.cc:288] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:52.412446 22816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3f9ea40edb4238aabfb1aa16a80a06" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" is_pre_election: true
I20260926 21:23:52.412475 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3f9ea40edb4238aabfb1aa16a80a06" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031" is_pre_election: true
I20260926 21:23:52.412771 22816 raft_consensus.cc:2454] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:52.412771 22677 raft_consensus.cc:2454] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:52.413261 22886 leader_election.cc:302] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21f356cfe3754636bb2ce9424b1b5ef1, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:52.413579 23037 raft_consensus.cc:2781] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:52.413676 23037 raft_consensus.cc:482] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:52.413801 23037 raft_consensus.cc:3037] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:52.415819 23037 raft_consensus.cc:504] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.416429 23037 leader_election.cc:288] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Requested vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:52.417014 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3f9ea40edb4238aabfb1aa16a80a06" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031"
I20260926 21:23:52.417014 22816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3f9ea40edb4238aabfb1aa16a80a06" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
I20260926 21:23:52.417273 22677 raft_consensus.cc:3037] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:52.417272 22816 raft_consensus.cc:3037] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:52.420174 22677 raft_consensus.cc:2454] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:52.420174 22816 raft_consensus.cc:2454] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:52.420719 22888 leader_election.cc:302] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21f356cfe3754636bb2ce9424b1b5ef1, 766ff97b80a3425590e5c8166cd2e031, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:52.421057 23037 raft_consensus.cc:2781] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:52.421274 23037 raft_consensus.cc:686] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Becoming Leader. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Running, Role: LEADER
I20260926 21:23:52.421629 23037 consensus_queue.cc:237] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.423418 23042 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: TxnStatusTablet state changed. Reason: New leader ce9fcec9c4ca456891c246e9ae5c875d. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.423776 23042 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:23:52.424400 23074 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 21:23:52.424531 23042 consensus_queue.cc:1035] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:52.424710 23074 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 21:23:52.425570 22518 catalog_manager.cc:5495] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3). New cstate: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } health_report { overall_health: HEALTHY } } }
I20260926 21:23:52.433503 23037 consensus_queue.cc:1035] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:52.530750 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.524213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43545 (local address 127.0.0.1:41720)
0926 21:23:52.524530 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:52.524579 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:52.524765 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:52.525414 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:52.525418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:52.525427 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:23:52.525591 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.525598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.527448 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.527453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.528428 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.528440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.529561 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.529565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.529686 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.529755 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:23:52.530265 (+ 510us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:23:52.530285 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 21:23:52.530366 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 21:23:52.530487 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 21:23:52.531024 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.524325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.14.142.3:43545)
0926 21:23:52.525045 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.525053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.525080 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:23:52.525159 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.525165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.525262 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.525381 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:52.525858 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.527281 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.528609 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.529375 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.529538 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.529955 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:52.530131 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:52.530234 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.530426 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.530528 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":390,"threads_started":1}
I20260926 21:23:52.535336 22677 raft_consensus.cc:1261] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Refusing update from remote peer ce9fcec9c4ca456891c246e9ae5c875d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:52.535339 22816 raft_consensus.cc:1261] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Refusing update from remote peer ce9fcec9c4ca456891c246e9ae5c875d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:52.536046 23072 consensus_queue.cc:1035] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:52.536473 23037 consensus_queue.cc:1035] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:52.541628 23038 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: TxnStatusTablet state changed. Reason: New leader ce9fcec9c4ca456891c246e9ae5c875d. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.541958 23038 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.542248 23039 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: TxnStatusTablet state changed. Reason: New leader ce9fcec9c4ca456891c246e9ae5c875d. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.542528 23039 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.545768 23077 mvcc.cc:205] Tried to move back new op lower bound from 7333715282057129984 to 7333715281601515520. Current Snapshot: MvccSnapshot[applied={T|T < 7333715282057129984}]
I20260926 21:23:52.546552 23080 mvcc.cc:205] Tried to move back new op lower bound from 7333715282057129984 to 7333715281601515520. Current Snapshot: MvccSnapshot[applied={T|T < 7333715282057129984}]
I20260926 21:23:52.553144 23037 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.553514 23037 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:23:52.555166 23037 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.555480 23037 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P ce9fcec9c4ca456891c246e9ae5c875d: This TxnStatusTablet replica's current role is: LEADER
I20260926 21:23:52.556370 23039 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.557708 23039 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P 21f356cfe3754636bb2ce9424b1b5ef1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.563122 23038 tablet_replica.cc:439] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } } }
I20260926 21:23:52.563423 23038 tablet_replica.cc:442] T ba3f9ea40edb4238aabfb1aa16a80a06 P 766ff97b80a3425590e5c8166cd2e031: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 21:23:52.581166 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.574176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48653 (local address 127.14.142.3:43545)
0926 21:23:52.574368 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.574374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.576149 (+ 1775us) server_negotiation.cc:407] Connection header received
0926 21:23:52.576380 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.576385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.576459 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.576557 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:52.577090 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.577862 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.579358 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.579790 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.579926 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.580334 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:52.580445 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:52.580546 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.580716 (+ 170us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.580828 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":66}
I20260926 21:23:52.581235 23090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.574099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43545 (local address 127.14.142.3:48653)
0926 21:23:52.576028 (+ 1929us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:52.576045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:52.576236 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:52.576594 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:52.576598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:52.576608 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 21:23:52.576909 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.576917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.578028 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.578036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.579158 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.579176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.579957 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.579963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.580110 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.580164 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 21:23:52.580543 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 21:23:52.580581 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:23:52.580696 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 21:23:52.580873 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1781,"mutex_wait_us":24,"thread_start_us":1694,"threads_started":1}
I20260926 21:23:52.658969 22518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:52.659248 22518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:52.662667 22518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Sending DeleteTablet for 3 replicas of tablet ff173e671f3f4b8e845e1aab80be7a54
I20260926 21:23:52.663636 22657 tablet_service.cc:1514] Processing DeleteTablet for tablet ff173e671f3f4b8e845e1aab80be7a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:52 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:57528
I20260926 21:23:52.663729 22795 tablet_service.cc:1514] Processing DeleteTablet for tablet ff173e671f3f4b8e845e1aab80be7a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:52 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:46364
I20260926 21:23:52.664335 23093 tablet_replica.cc:330] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: stopping tablet replica
I20260926 21:23:52.664638 23093 raft_consensus.cc:2229] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:52.664572 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet ff173e671f3f4b8e845e1aab80be7a54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:52 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41700
I20260926 21:23:52.665048 23093 raft_consensus.cc:2258] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:52.666671 23093 ts_tablet_manager.cc:1905] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:52.678180 23093 ts_tablet_manager.cc:1918] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:52.678546 23093 log.cc:1196] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/ff173e671f3f4b8e845e1aab80be7a54
I20260926 21:23:52.679347 23093 ts_tablet_manager.cc:1939] T ff173e671f3f4b8e845e1aab80be7a54 P 766ff97b80a3425590e5c8166cd2e031: Deleting consensus metadata
I20260926 21:23:52.681077 22506 catalog_manager.cc:4841] TS 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475): tablet ff173e671f3f4b8e845e1aab80be7a54 (table test-table [id=2232033837f24c44afb1c61c12f3e47b]) successfully deleted
I20260926 21:23:52.681527 23094 tablet_replica.cc:330] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: stopping tablet replica
I20260926 21:23:52.681970 23094 raft_consensus.cc:2229] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:52.682418 23094 raft_consensus.cc:2258] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:52.684118 23094 ts_tablet_manager.cc:1905] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:52.689779 23095 tablet_replica.cc:330] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: stopping tablet replica
I20260926 21:23:52.690111 23095 raft_consensus.cc:2229] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:52.690665 23095 raft_consensus.cc:2258] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:52.691992 23095 ts_tablet_manager.cc:1905] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:52.695433 23094 ts_tablet_manager.cc:1918] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:52.695596 23094 log.cc:1196] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/ff173e671f3f4b8e845e1aab80be7a54
I20260926 21:23:52.696100 23094 ts_tablet_manager.cc:1939] T ff173e671f3f4b8e845e1aab80be7a54 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting consensus metadata
I20260926 21:23:52.697479 22504 catalog_manager.cc:4841] TS 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441): tablet ff173e671f3f4b8e845e1aab80be7a54 (table test-table [id=2232033837f24c44afb1c61c12f3e47b]) successfully deleted
I20260926 21:23:52.699795 23095 ts_tablet_manager.cc:1918] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:52.699961 23095 log.cc:1196] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/ff173e671f3f4b8e845e1aab80be7a54
I20260926 21:23:52.700419 23095 ts_tablet_manager.cc:1939] T ff173e671f3f4b8e845e1aab80be7a54 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting consensus metadata
I20260926 21:23:52.701598 22507 catalog_manager.cc:4841] TS ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545): tablet ff173e671f3f4b8e845e1aab80be7a54 (table test-table [id=2232033837f24c44afb1c61c12f3e47b]) successfully deleted
Sep 26 21:23:52 dist-test-slave-4d5t krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 21:23:52 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:23:52 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 21:23:52.746120 23103 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:23:52.732511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.14.142.62:34439)
0926 21:23:52.733443 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.733448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.733467 (+ 19us) server_negotiation.cc:407] Connection header received
0926 21:23:52.733524 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.733527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.733612 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.733708 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:52.735632 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.736737 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.738605 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.739050 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.739176 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.745810 (+ 6634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55818: BlockingRecv error: recv got EOF from 127.0.0.1:55818 (error 108)
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":628,"threads_started":1}
Sep 26 21:23:52 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM
I20260926 21:23:52.767014 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.749222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.14.142.62:34439)
0926 21:23:52.749434 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.749440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.749789 (+ 349us) server_negotiation.cc:407] Connection header received
0926 21:23:52.749958 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.749963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.750030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.750126 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:52.751929 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.753109 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.754978 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.755477 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.755594 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.760816 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:52.760822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:52.760825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:52.760878 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:52.763560 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:52.764819 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:52.764824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:52.764827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:52.764999 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:52.765480 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:52.765484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:52.765486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:52.765839 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:52.766084 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.766475 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.766671 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 21:23:52.771833 22518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:52.772523 22518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:52.780404 22657 tablet_service.cc:1467] Processing CreateTablet for tablet d404503d95c848ca97506fa32999b743 (DEFAULT_TABLE table=test-table [id=48c008d29ab142e39c6c5d9987fabed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:52.780525 22795 tablet_service.cc:1467] Processing CreateTablet for tablet d404503d95c848ca97506fa32999b743 (DEFAULT_TABLE table=test-table [id=48c008d29ab142e39c6c5d9987fabed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:52.780974 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d404503d95c848ca97506fa32999b743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:52.781044 22795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d404503d95c848ca97506fa32999b743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:52.785431 22936 tablet_service.cc:1467] Processing CreateTablet for tablet d404503d95c848ca97506fa32999b743 (DEFAULT_TABLE table=test-table [id=48c008d29ab142e39c6c5d9987fabed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:52.786216 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d404503d95c848ca97506fa32999b743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:52.797694 23110 tablet_bootstrap.cc:492] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Bootstrap starting.
I20260926 21:23:52.798311 23109 tablet_bootstrap.cc:492] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Bootstrap starting.
I20260926 21:23:52.798597 23111 tablet_bootstrap.cc:492] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Bootstrap starting.
I20260926 21:23:52.800130 23109 tablet_bootstrap.cc:654] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:52.800128 23110 tablet_bootstrap.cc:654] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:52.800693 23111 tablet_bootstrap.cc:654] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:52.802663 23109 tablet_bootstrap.cc:492] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: No bootstrap required, opened a new log
I20260926 21:23:52.802829 23109 ts_tablet_manager.cc:1397] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 21:23:52.803722 23109 raft_consensus.cc:348] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.804024 23109 raft_consensus.cc:374] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:52.804131 23109 raft_consensus.cc:729] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f356cfe3754636bb2ce9424b1b5ef1, State: Initialized, Role: FOLLOWER
I20260926 21:23:52.804423 23109 consensus_queue.cc:260] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.807765 23109 ts_tablet_manager.cc:1428] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260926 21:23:52.810626 23110 tablet_bootstrap.cc:492] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: No bootstrap required, opened a new log
I20260926 21:23:52.810858 23110 ts_tablet_manager.cc:1397] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260926 21:23:52.811718 23110 raft_consensus.cc:348] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.811957 23110 raft_consensus.cc:374] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:52.812075 23110 raft_consensus.cc:729] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766ff97b80a3425590e5c8166cd2e031, State: Initialized, Role: FOLLOWER
I20260926 21:23:52.812086 23111 tablet_bootstrap.cc:492] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: No bootstrap required, opened a new log
I20260926 21:23:52.812254 23111 ts_tablet_manager.cc:1397] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.012s
I20260926 21:23:52.812389 23110 consensus_queue.cc:260] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.813215 23111 raft_consensus.cc:348] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.813478 23111 raft_consensus.cc:374] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:52.813577 23111 raft_consensus.cc:729] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Initialized, Role: FOLLOWER
I20260926 21:23:52.813639 23110 ts_tablet_manager.cc:1428] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 21:23:52.813848 23111 consensus_queue.cc:260] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.814628 23111 ts_tablet_manager.cc:1428] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 21:23:52.876873 23039 raft_consensus.cc:482] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:52.877170 23039 raft_consensus.cc:504] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.879194 23039 leader_election.cc:288] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545)
I20260926 21:23:52.897496 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.879631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39475 (local address 127.14.142.2:53011)
0926 21:23:52.890491 (+ 10860us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:52.890516 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:52.890680 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:52.891926 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:52.891931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:52.891963 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:52.892326 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.892335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.893581 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.893597 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.895213 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.895226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.896584 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.896592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.896771 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.896807 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:23:52.896911 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 21:23:52.897053 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10721,"mutex_wait_us":67,"thread_start_us":10723,"threads_started":1}
I20260926 21:23:52.897511 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.879670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:53011 (local address 127.14.142.1:39475)
0926 21:23:52.891579 (+ 11909us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.891596 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.891621 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:23:52.891694 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.891698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.891771 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.891906 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:52.892605 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.893428 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.895372 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.896335 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.896541 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.896645 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.896960 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.897111 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11772,"thread_start_us":11502,"threads_started":1}
I20260926 21:23:52.898602 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.879500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:43545 (local address 127.14.142.2:42511)
0926 21:23:52.890656 (+ 11156us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:52.890671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:52.890785 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:52.891502 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:52.891507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:52.891539 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:23:52.891919 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.891934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.893619 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.893625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.895857 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:52.895871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.897825 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.897831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:52.897998 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.898031 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 21:23:52.898129 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 21:23:52.898250 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10740,"thread_start_us":11239,"threads_started":1}
I20260926 21:23:52.898489 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d404503d95c848ca97506fa32999b743" candidate_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031" is_pre_election: true
I20260926 21:23:52.898821 22676 raft_consensus.cc:2454] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21f356cfe3754636bb2ce9424b1b5ef1 in term 0.
I20260926 21:23:52.898635 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.880051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:42511 (local address 127.14.142.3:43545)
0926 21:23:52.880298 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.880308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.890745 (+ 10437us) server_negotiation.cc:407] Connection header received
0926 21:23:52.890907 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.891031 (+ 124us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.891232 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.891485 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:23:52.892137 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.893417 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.896040 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.897352 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.897530 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.897818 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.898179 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.898320 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 21:23:52.899379 22749 leader_election.cc:302] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21f356cfe3754636bb2ce9424b1b5ef1, 766ff97b80a3425590e5c8166cd2e031; no voters:
I20260926 21:23:52.899860 23039 raft_consensus.cc:2781] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:52.899988 23039 raft_consensus.cc:482] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:52.900087 23039 raft_consensus.cc:3037] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:52.900031 22961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d404503d95c848ca97506fa32999b743" candidate_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" is_pre_election: true
I20260926 21:23:52.900512 22961 raft_consensus.cc:2454] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21f356cfe3754636bb2ce9424b1b5ef1 in term 0.
I20260926 21:23:52.902683 23039 raft_consensus.cc:504] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.903332 23039 leader_election.cc:288] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [CANDIDATE]: Term 1 election: Requested vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545)
I20260926 21:23:52.903914 22676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d404503d95c848ca97506fa32999b743" candidate_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031"
I20260926 21:23:52.903983 22961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d404503d95c848ca97506fa32999b743" candidate_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9fcec9c4ca456891c246e9ae5c875d"
I20260926 21:23:52.904141 22676 raft_consensus.cc:3037] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:52.904220 22961 raft_consensus.cc:3037] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:52.906445 22676 raft_consensus.cc:2454] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21f356cfe3754636bb2ce9424b1b5ef1 in term 1.
I20260926 21:23:52.906903 22749 leader_election.cc:302] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21f356cfe3754636bb2ce9424b1b5ef1, 766ff97b80a3425590e5c8166cd2e031; no voters:
I20260926 21:23:52.907210 23039 raft_consensus.cc:2781] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:52.907451 22961 raft_consensus.cc:2454] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21f356cfe3754636bb2ce9424b1b5ef1 in term 1.
I20260926 21:23:52.908026 23039 raft_consensus.cc:686] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 LEADER]: Becoming Leader. State: Replica: 21f356cfe3754636bb2ce9424b1b5ef1, State: Running, Role: LEADER
I20260926 21:23:52.908425 23039 consensus_queue.cc:237] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:52.912235 22519 catalog_manager.cc:5495] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2). New cstate: current_term: 1 leader_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } health_report { overall_health: UNKNOWN } } }
I20260926 21:23:52.956583 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:52.949679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.14.142.2:38441)
0926 21:23:52.950274 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:52.950279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:52.950302 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:23:52.950396 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:52.950400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:52.950481 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:52.950643 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:52.951005 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.951924 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.953724 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:52.954323 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:52.954452 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:52.954923 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:52.955635 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:52.955757 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:52.956001 (+ 244us) server_negotiation.cc:299] Negotiation successful
0926 21:23:52.956129 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":353,"threads_started":1}
I20260926 21:23:52.960469 22676 raft_consensus.cc:1261] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Refusing update from remote peer 21f356cfe3754636bb2ce9424b1b5ef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:52.960788 22961 raft_consensus.cc:1261] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Refusing update from remote peer 21f356cfe3754636bb2ce9424b1b5ef1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:52.961336 23118 consensus_queue.cc:1035] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:52.961755 23039 consensus_queue.cc:1035] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:52.983814 22519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:52.984107 22519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:52.988083 22519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Sending DeleteTablet for 3 replicas of tablet d404503d95c848ca97506fa32999b743
I20260926 21:23:52.989101 22657 tablet_service.cc:1514] Processing DeleteTablet for tablet d404503d95c848ca97506fa32999b743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:52 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:57528
I20260926 21:23:52.989435 23093 tablet_replica.cc:330] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: stopping tablet replica
I20260926 21:23:52.989460 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet d404503d95c848ca97506fa32999b743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:52 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41700
I20260926 21:23:52.989661 23093 raft_consensus.cc:2229] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:52.989741 23095 tablet_replica.cc:330] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: stopping tablet replica
I20260926 21:23:52.989916 23093 raft_consensus.cc:2258] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:52.989928 23095 raft_consensus.cc:2229] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:52.990140 23095 raft_consensus.cc:2258] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:52.991905 23093 ts_tablet_manager.cc:1905] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:52.992690 23095 ts_tablet_manager.cc:1905] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:52.997005 22519 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:23:52.998378 22519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:23:53.000077 22795 tablet_service.cc:1514] Processing DeleteTablet for tablet d404503d95c848ca97506fa32999b743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:52 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:46364
I20260926 21:23:53.000454 23094 tablet_replica.cc:330] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: stopping tablet replica
I20260926 21:23:53.000640 23093 ts_tablet_manager.cc:1918] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:53.000725 23094 raft_consensus.cc:2229] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:53.000814 23093 log.cc:1196] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/d404503d95c848ca97506fa32999b743
I20260926 21:23:53.001331 23094 raft_consensus.cc:2258] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:53.003072 23094 ts_tablet_manager.cc:1905] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:53.005666 23095 ts_tablet_manager.cc:1918] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:53.005926 23095 log.cc:1196] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/d404503d95c848ca97506fa32999b743
I20260926 21:23:53.006464 23095 ts_tablet_manager.cc:1939] T d404503d95c848ca97506fa32999b743 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting consensus metadata
I20260926 21:23:53.008319 22507 catalog_manager.cc:4841] TS ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545): tablet d404503d95c848ca97506fa32999b743 (table test-table [id=48c008d29ab142e39c6c5d9987fabed6]) successfully deleted
I20260926 21:23:53.008514 22657 tablet_service.cc:1467] Processing CreateTablet for tablet b447c1d64ede49efa8f1122c62d55361 (DEFAULT_TABLE table=test-table [id=d0bbdda95d04487eb549acca5a6d7868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:53.009091 22657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b447c1d64ede49efa8f1122c62d55361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:53.009262 22795 tablet_service.cc:1467] Processing CreateTablet for tablet b447c1d64ede49efa8f1122c62d55361 (DEFAULT_TABLE table=test-table [id=d0bbdda95d04487eb549acca5a6d7868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:53.009737 22795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b447c1d64ede49efa8f1122c62d55361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:53.011190 22936 tablet_service.cc:1467] Processing CreateTablet for tablet b447c1d64ede49efa8f1122c62d55361 (DEFAULT_TABLE table=test-table [id=d0bbdda95d04487eb549acca5a6d7868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:23:53.011658 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b447c1d64ede49efa8f1122c62d55361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:53.013299 23094 ts_tablet_manager.cc:1918] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:23:53.013478 23094 log.cc:1196] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/d404503d95c848ca97506fa32999b743
I20260926 21:23:53.014048 23094 ts_tablet_manager.cc:1939] T d404503d95c848ca97506fa32999b743 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting consensus metadata
I20260926 21:23:53.017938 23109 tablet_bootstrap.cc:492] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Bootstrap starting.
I20260926 21:23:53.018187 22504 catalog_manager.cc:4841] TS 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441): tablet d404503d95c848ca97506fa32999b743 (table test-table [id=48c008d29ab142e39c6c5d9987fabed6]) successfully deleted
I20260926 21:23:53.019577 23093 ts_tablet_manager.cc:1939] T d404503d95c848ca97506fa32999b743 P 766ff97b80a3425590e5c8166cd2e031: Deleting consensus metadata
I20260926 21:23:53.019852 23109 tablet_bootstrap.cc:654] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:53.020871 22506 catalog_manager.cc:4841] TS 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475): tablet d404503d95c848ca97506fa32999b743 (table test-table [id=48c008d29ab142e39c6c5d9987fabed6]) successfully deleted
I20260926 21:23:53.026520 23111 tablet_bootstrap.cc:492] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Bootstrap starting.
I20260926 21:23:53.028846 23111 tablet_bootstrap.cc:654] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:53.030361 23110 tablet_bootstrap.cc:492] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Bootstrap starting.
I20260926 21:23:53.031708 23109 tablet_bootstrap.cc:492] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: No bootstrap required, opened a new log
I20260926 21:23:53.031986 23109 ts_tablet_manager.cc:1397] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260926 21:23:53.032929 23109 raft_consensus.cc:348] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.033324 23109 raft_consensus.cc:374] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:53.033468 23109 raft_consensus.cc:729] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21f356cfe3754636bb2ce9424b1b5ef1, State: Initialized, Role: FOLLOWER
I20260926 21:23:53.033885 23109 consensus_queue.cc:260] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.034379 23110 tablet_bootstrap.cc:654] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:53.034631 23109 ts_tablet_manager.cc:1428] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 21:23:53.036346 23111 tablet_bootstrap.cc:492] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: No bootstrap required, opened a new log
I20260926 21:23:53.036643 23111 ts_tablet_manager.cc:1397] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260926 21:23:53.037503 23111 raft_consensus.cc:348] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.037846 23111 raft_consensus.cc:374] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:53.037968 23111 raft_consensus.cc:729] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Initialized, Role: FOLLOWER
I20260926 21:23:53.038270 23111 consensus_queue.cc:260] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.038903 23111 ts_tablet_manager.cc:1428] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 21:23:53.041260 23110 tablet_bootstrap.cc:492] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: No bootstrap required, opened a new log
I20260926 21:23:53.041489 23110 ts_tablet_manager.cc:1397] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.007s
I20260926 21:23:53.042349 23110 raft_consensus.cc:348] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.042654 23110 raft_consensus.cc:374] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:53.042749 23110 raft_consensus.cc:729] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766ff97b80a3425590e5c8166cd2e031, State: Initialized, Role: FOLLOWER
I20260926 21:23:53.043005 23110 consensus_queue.cc:260] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.043515 23110 ts_tablet_manager.cc:1428] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 21:23:53.175448 23072 raft_consensus.cc:482] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:23:53.175719 23072 raft_consensus.cc:504] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.176506 23072 leader_election.cc:288] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:53.177199 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b447c1d64ede49efa8f1122c62d55361" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031" is_pre_election: true
I20260926 21:23:53.177211 22815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b447c1d64ede49efa8f1122c62d55361" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" is_pre_election: true
I20260926 21:23:53.177497 22677 raft_consensus.cc:2454] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:53.177525 22815 raft_consensus.cc:2454] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 0.
I20260926 21:23:53.177989 22888 leader_election.cc:302] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766ff97b80a3425590e5c8166cd2e031, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:53.178270 23072 raft_consensus.cc:2781] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:23:53.178375 23072 raft_consensus.cc:482] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:23:53.178498 23072 raft_consensus.cc:3037] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:53.180537 23072 raft_consensus.cc:504] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.181295 23072 leader_election.cc:288] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Requested vote from peers 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475), 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441)
I20260926 21:23:53.181825 22677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b447c1d64ede49efa8f1122c62d55361" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766ff97b80a3425590e5c8166cd2e031"
I20260926 21:23:53.181922 22815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b447c1d64ede49efa8f1122c62d55361" candidate_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21f356cfe3754636bb2ce9424b1b5ef1"
I20260926 21:23:53.182096 22677 raft_consensus.cc:3037] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:53.182144 22815 raft_consensus.cc:3037] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:53.185252 22677 raft_consensus.cc:2454] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:53.185349 22815 raft_consensus.cc:2454] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce9fcec9c4ca456891c246e9ae5c875d in term 1.
I20260926 21:23:53.185736 22888 leader_election.cc:302] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766ff97b80a3425590e5c8166cd2e031, ce9fcec9c4ca456891c246e9ae5c875d; no voters:
I20260926 21:23:53.186045 23072 raft_consensus.cc:2781] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:53.186472 23072 raft_consensus.cc:686] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Becoming Leader. State: Replica: ce9fcec9c4ca456891c246e9ae5c875d, State: Running, Role: LEADER
I20260926 21:23:53.187089 23072 consensus_queue.cc:237] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } }
I20260926 21:23:53.190117 22519 catalog_manager.cc:5495] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d reported cstate change: term changed from 0 to 1, leader changed from <none> to ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3). New cstate: current_term: 1 leader_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce9fcec9c4ca456891c246e9ae5c875d" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 43545 } health_report { overall_health: HEALTHY } } }
I20260926 21:23:53.250029 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:53.242689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.14.142.3:43545)
0926 21:23:53.242913 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:53.242922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:53.242947 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:23:53.243137 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:53.243145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:53.243253 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:53.243373 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:23:53.243781 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.245023 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.246947 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.247888 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.248041 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:53.248800 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:23:53.249086 (+ 286us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:23:53.249253 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:53.249419 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 21:23:53.249582 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:23:53.262645 22677 raft_consensus.cc:1261] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Refusing update from remote peer ce9fcec9c4ca456891c246e9ae5c875d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:53.262931 22815 raft_consensus.cc:1261] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Refusing update from remote peer ce9fcec9c4ca456891c246e9ae5c875d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:23:53.263454 23129 consensus_queue.cc:1035] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "766ff97b80a3425590e5c8166cd2e031" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:53.263839 23072 consensus_queue.cc:1035] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [LEADER]: Connected to new peer: Peer: permanent_uuid: "21f356cfe3754636bb2ce9424b1b5ef1" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:23:53.346517 22519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:53.346979 22519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:23:53.350869 22519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd5a0229b00541919c276cf8421bde90: Sending DeleteTablet for 3 replicas of tablet b447c1d64ede49efa8f1122c62d55361
I20260926 21:23:53.351975 22795 tablet_service.cc:1514] Processing DeleteTablet for tablet b447c1d64ede49efa8f1122c62d55361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:53 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:46364
I20260926 21:23:53.351949 22657 tablet_service.cc:1514] Processing DeleteTablet for tablet b447c1d64ede49efa8f1122c62d55361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:53 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:57528
I20260926 21:23:53.352038 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet b447c1d64ede49efa8f1122c62d55361 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:23:53 UTC) from {username='oryx', principal='oryx/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41700
I20260926 21:23:53.352321 23094 tablet_replica.cc:330] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: stopping tablet replica
I20260926 21:23:53.352340 23093 tablet_replica.cc:330] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: stopping tablet replica
I20260926 21:23:53.352517 23093 raft_consensus.cc:2229] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:53.352530 23094 raft_consensus.cc:2229] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:23:53.352770 23093 raft_consensus.cc:2258] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:53.352777 23094 raft_consensus.cc:2258] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:53.353099 23095 tablet_replica.cc:330] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: stopping tablet replica
I20260926 21:23:53.353296 23095 raft_consensus.cc:2229] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:23:53.353741 23095 raft_consensus.cc:2258] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:23:53.354080 23094 ts_tablet_manager.cc:1905] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:53.354305 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 22591
I20260926 21:23:53.355371 23093 ts_tablet_manager.cc:1905] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:53.357707 23095 ts_tablet_manager.cc:1905] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:23:53.362531 23093 ts_tablet_manager.cc:1918] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:53.362677 23093 log.cc:1196] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/b447c1d64ede49efa8f1122c62d55361
I20260926 21:23:53.363099 23093 ts_tablet_manager.cc:1939] T b447c1d64ede49efa8f1122c62d55361 P 766ff97b80a3425590e5c8166cd2e031: Deleting consensus metadata
I20260926 21:23:53.364260 22506 catalog_manager.cc:4841] TS 766ff97b80a3425590e5c8166cd2e031 (127.14.142.1:39475): tablet b447c1d64ede49efa8f1122c62d55361 (table test-table [id=d0bbdda95d04487eb549acca5a6d7868]) successfully deleted
I20260926 21:23:53.364794 23095 ts_tablet_manager.cc:1918] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:53.364930 23095 log.cc:1196] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/b447c1d64ede49efa8f1122c62d55361
I20260926 21:23:53.365212 23094 ts_tablet_manager.cc:1918] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 21:23:53.365382 23095 ts_tablet_manager.cc:1939] T b447c1d64ede49efa8f1122c62d55361 P ce9fcec9c4ca456891c246e9ae5c875d: Deleting consensus metadata
I20260926 21:23:53.365387 23094 log.cc:1196] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/b447c1d64ede49efa8f1122c62d55361
I20260926 21:23:53.365839 23094 ts_tablet_manager.cc:1939] T b447c1d64ede49efa8f1122c62d55361 P 21f356cfe3754636bb2ce9424b1b5ef1: Deleting consensus metadata
I20260926 21:23:53.366642 22507 catalog_manager.cc:4841] TS ce9fcec9c4ca456891c246e9ae5c875d (127.14.142.3:43545): tablet b447c1d64ede49efa8f1122c62d55361 (table test-table [id=d0bbdda95d04487eb549acca5a6d7868]) successfully deleted
I20260926 21:23:53.366937 22504 catalog_manager.cc:4841] TS 21f356cfe3754636bb2ce9424b1b5ef1 (127.14.142.2:38441): tablet b447c1d64ede49efa8f1122c62d55361 (table test-table [id=d0bbdda95d04487eb549acca5a6d7868]) successfully deleted
Sep 26 21:23:53 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.1@KRBTEST.COM
I20260926 21:23:53.377025 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:53.356957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.14.142.1:39475)
0926 21:23:53.357166 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:53.357173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:53.357999 (+ 826us) server_negotiation.cc:407] Connection header received
0926 21:23:53.358319 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:53.358325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:53.358412 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:53.358484 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:53.360285 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.361412 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.363376 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.364042 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.364138 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:53.370520 (+ 6382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:53.370538 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:53.370555 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:53.370604 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:53.373900 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:53.374705 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:53.374712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:53.374715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:53.374883 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:53.375290 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:53.375298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:53.375301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:53.375723 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:53.376053 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:53.376373 (+ 320us) server_negotiation.cc:299] Negotiation successful
0926 21:23:53.376681 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 21:23:53.377784 22722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:53.485423 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 22591
I20260926 21:23:53.512360 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 22731
Sep 26 21:23:53 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.2@KRBTEST.COM
I20260926 21:23:53.529958 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:53.513347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.14.142.2:38441)
0926 21:23:53.513870 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:53.513874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:53.513892 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:23:53.513944 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:53.513949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:53.514008 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:53.514137 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:53.515359 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.516129 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.517516 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.517940 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.518079 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:53.522869 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:53.522881 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:53.522898 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:53.522936 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:53.526604 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:53.527459 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:53.527474 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:53.527476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:53.527619 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:53.528030 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:53.528036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:53.528039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:53.528453 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:53.528870 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:53.529274 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 21:23:53.529535 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":278,"threads_started":1}
I20260926 21:23:53.530714 22861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:53.657510 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 22731
I20260926 21:23:53.681607 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 22870
Sep 26 21:23:53 dist-test-slave-4d5t krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.14.142.3@KRBTEST.COM
I20260926 21:23:53.699434 23082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:53.682558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.14.142.3:43545)
0926 21:23:53.682736 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:53.682741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:53.682761 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:23:53.682929 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:53.682933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:53.683000 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:53.683109 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:53.684770 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.685582 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.687188 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:53.687641 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:53.687759 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:53.692515 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:53.692531 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:53.692609 (+ 78us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:53.692667 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:53.696763 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:53.697517 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:53.697524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:53.697526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:53.697675 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:53.698020 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:53.698023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:53.698026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:53.698419 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:53.698688 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:53.698934 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 21:23:53.699131 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 21:23:53.700217 23001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:53.857508 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 22870
I20260926 21:23:53.884523 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 22488
I20260926 21:23:53.885987 22560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:54.008947 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 22488
2026-09-26T21:23:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7586 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:54 dist-test-slave-4d5t krb5kdc[23145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:54 dist-test-slave-4d5t krb5kdc[23145](info): set up 2 sockets
Sep 26 21:23:54 dist-test-slave-4d5t krb5kdc[23145](info): commencing operation
krb5kdc: starting...
W20260926 21:23:55.189908 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:55 dist-test-slave-4d5t krb5kdc[23145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:55.437243 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41585
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:55.784402 23158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:55.784801 23158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:55.784991 23158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:55.792858 23158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:55.793006 23158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:55.793087 23158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:55.793182 23158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:55.793249 23158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:55.804714 23158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:55.806118 23158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:55.807687 23158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:55.820391 23163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:55.820355 23164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:55.820429 23166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:55.821187 23158 server_base.cc:1034] running on GCE node
I20260926 21:23:55.822096 23158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:55.823722 23158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:55.824993 23158 hybrid_clock.cc:648] HybridClock initialized: now 1790457835824959 us; error 64 us; skew 500 ppm
I20260926 21:23:55.825404 23158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T21:23:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1882 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:23:56 dist-test-slave-4d5t krb5kdc[23172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:23:56 dist-test-slave-4d5t krb5kdc[23172](info): set up 2 sockets
Sep 26 21:23:56 dist-test-slave-4d5t krb5kdc[23172](info): commencing operation
krb5kdc: starting...
W20260926 21:23:57.109925 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:23:57 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:23:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:23:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:23:57.357738 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:37799
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:37799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:57.715801 23185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:57.716208 23185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:57.716398 23185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:57.724875 23185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:23:57.725111 23185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:57.725262 23185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:57.725374 23185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:23:57.725463 23185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:23:57.735929 23185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:37799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:57.737413 23185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:57.738781 23185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:57.751452 23190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:57.751623 23193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:57.751813 23191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:57.752374 23185 server_base.cc:1034] running on GCE node
I20260926 21:23:57.753297 23185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:57.754899 23185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:57.756247 23185 hybrid_clock.cc:648] HybridClock initialized: now 1790457837756166 us; error 54 us; skew 500 ppm
I20260926 21:23:57.757019 23185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:57 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457837, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:57.767866 23185 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:23:57.769730 23185 webserver.cc:460] Webserver started at http://127.14.142.62:42029/ using document root <none> and password file <none>
I20260926 21:23:57.770481 23185 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:57.770628 23185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:57.770951 23185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:57.774663 23185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "e416f3c972bd4f15906edc9782d8978b"
format_stamp: "Formatted at 2026-09-26 21:23:57 on dist-test-slave-4d5t"
server_key: "36551124c462c58630f0b48d69d68743"
server_key_iv: "e7a4db2728ea0f5badbcf8d66ab9a679"
server_key_version: "encryptionkey@0"
I20260926 21:23:57.775466 23185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "e416f3c972bd4f15906edc9782d8978b"
format_stamp: "Formatted at 2026-09-26 21:23:57 on dist-test-slave-4d5t"
server_key: "36551124c462c58630f0b48d69d68743"
server_key_iv: "e7a4db2728ea0f5badbcf8d66ab9a679"
server_key_version: "encryptionkey@0"
I20260926 21:23:57.781383 23185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260926 21:23:57.786038 23200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:57.788043 23185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260926 21:23:57.788297 23185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "e416f3c972bd4f15906edc9782d8978b"
format_stamp: "Formatted at 2026-09-26 21:23:57 on dist-test-slave-4d5t"
server_key: "36551124c462c58630f0b48d69d68743"
server_key_iv: "e7a4db2728ea0f5badbcf8d66ab9a679"
server_key_version: "encryptionkey@0"
I20260926 21:23:57.788558 23185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:57.807273 23185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:57.811820 23185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:57.812116 23185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:57.834038 23185 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:37799
I20260926 21:23:57.834039 23251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:37799 every 8 connection(s)
I20260926 21:23:57.835570 23185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:23:57.839812 23252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:23:57.844027 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23185
I20260926 21:23:57.844311 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:23:57.845237 14904 external_mini_cluster.cc:1428] Setting key 1c7f3b0eee48efac1ada9ea743fcad69
I20260926 21:23:57.854112 23252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Bootstrap starting.
I20260926 21:23:57.857393 23252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Neither blocks nor log segments found. Creating new log.
I20260926 21:23:57.858575 23252 log.cc:824] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Log is configured to *not* fsync() on all Append() calls
Sep 26 21:23:57 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:57.861712 23252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: No bootstrap required, opened a new log
I20260926 21:23:57.869997 23252 raft_consensus.cc:348] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:23:57.870419 23252 raft_consensus.cc:374] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:23:57.870537 23252 raft_consensus.cc:729] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e416f3c972bd4f15906edc9782d8978b, State: Initialized, Role: FOLLOWER
I20260926 21:23:57.870743 23255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:57.847627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.14.142.62:37799)
0926 21:23:57.848452 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:57.848464 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:57.848507 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:23:57.850358 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:57.850382 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:57.850826 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:57.851614 (+ 788us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:57.853483 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:57.854890 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:57.856165 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:57.856907 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:57.857099 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:57.862101 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:57.862109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:57.862129 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:57.862168 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:57.865919 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:57.866833 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:57.866840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:57.866847 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:57.867064 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:57.867475 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:57.867482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:57.867486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:57.868130 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:57.868502 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:57.869051 (+ 549us) server_negotiation.cc:299] Negotiation successful
0926 21:23:57.869504 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":370,"threads_started":1}
I20260926 21:23:57.871217 23252 consensus_queue.cc:260] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:23:57.871486 23252 raft_consensus.cc:388] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:23:57.871637 23252 raft_consensus.cc:482] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:23:57.871811 23252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:23:57.874415 23252 raft_consensus.cc:504] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:23:57.874939 23252 leader_election.cc:302] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e416f3c972bd4f15906edc9782d8978b; no voters:
I20260926 21:23:57.875547 23252 leader_election.cc:288] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:23:57.875806 23257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:23:57.876603 23257 raft_consensus.cc:686] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 LEADER]: Becoming Leader. State: Replica: e416f3c972bd4f15906edc9782d8978b, State: Running, Role: LEADER
I20260926 21:23:57.877105 23257 consensus_queue.cc:237] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:23:57.877712 23252 sys_catalog.cc:564] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:23:57.879894 23259 sys_catalog.cc:455] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e416f3c972bd4f15906edc9782d8978b. Latest consensus state: current_term: 1 leader_uuid: "e416f3c972bd4f15906edc9782d8978b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } } }
I20260926 21:23:57.880353 23259 sys_catalog.cc:458] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:57.882378 23258 sys_catalog.cc:455] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e416f3c972bd4f15906edc9782d8978b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } } }
I20260926 21:23:57.882702 23258 sys_catalog.cc:458] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: This master's current role is: LEADER
I20260926 21:23:57.882984 23263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:23:57.888518 23263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:23:57.900458 23263 catalog_manager.cc:1347] Generated new cluster ID: 3fd4ed6e3f384a59a4e733f0ec0afa38
I20260926 21:23:57.900622 23263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:23:57.920763 23263 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:23:57.922152 23263 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:23:57.937145 23263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Generated new TSK 0
I20260926 21:23:57.938114 23263 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:23:58.013952 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37799
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:58.367698 23278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:58.368046 23278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:58.368216 23278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:58.375615 23278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:58.375792 23278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:58.375979 23278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:23:58.387090 23278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:58.388474 23278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:58.389825 23278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:58.404740 23286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:58.404914 23283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:58.404894 23284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:58.406563 23278 server_base.cc:1034] running on GCE node
I20260926 21:23:58.407492 23278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:58.408872 23278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:58.410154 23278 hybrid_clock.cc:648] HybridClock initialized: now 1790457838410083 us; error 45 us; skew 500 ppm
I20260926 21:23:58.410627 23278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:58 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:58.414347 23278 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:23:58.415927 23278 webserver.cc:460] Webserver started at http://127.14.142.1:33709/ using document root <none> and password file <none>
I20260926 21:23:58.416795 23278 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:58.416911 23278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:58.417209 23278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:58.420327 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "f1c8c5e205c64d5db54fe65ef286d243"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "2db0e1a230258698a710ab57171f1304"
server_key_iv: "22e6841c211ce6fd90616f9915e50523"
server_key_version: "encryptionkey@0"
I20260926 21:23:58.421319 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "f1c8c5e205c64d5db54fe65ef286d243"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "2db0e1a230258698a710ab57171f1304"
server_key_iv: "22e6841c211ce6fd90616f9915e50523"
server_key_version: "encryptionkey@0"
I20260926 21:23:58.427021 23278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:23:58.430629 23293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:58.432248 23278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:23:58.432595 23278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "f1c8c5e205c64d5db54fe65ef286d243"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "2db0e1a230258698a710ab57171f1304"
server_key_iv: "22e6841c211ce6fd90616f9915e50523"
server_key_version: "encryptionkey@0"
I20260926 21:23:58.432828 23278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:58.462448 23278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:58.468685 23278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:58.469308 23278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:58.470986 23278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:58.473697 23278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:58.473801 23278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:58.473942 23278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:58.474014 23278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:58.521806 23278 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:44803
I20260926 21:23:58.521858 23405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:44803 every 8 connection(s)
I20260926 21:23:58.523777 23278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:23:58.531186 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23278
I20260926 21:23:58.531476 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:23:58.532250 14904 external_mini_cluster.cc:1428] Setting key 079acb881a0facb28d3a817d3d35392e
Sep 26 21:23:58 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:58.552141 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:58.528306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:48951 (local address 127.14.142.62:37799)
0926 21:23:58.528858 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:58.528863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:58.529896 (+ 1033us) server_negotiation.cc:407] Connection header received
0926 21:23:58.532601 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:58.532608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:58.532712 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:58.532813 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:58.534399 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:58.535325 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:58.538483 (+ 3158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:58.539288 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:58.539451 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:58.544399 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:58.544406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:58.544409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:58.544445 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:58.547592 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:58.548982 (+ 1390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:58.548988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:58.548991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:58.549291 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:58.549863 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:58.549868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:58.549871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:58.550231 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:58.550374 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:58.551441 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0926 21:23:58.551834 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":314,"threads_started":1}
I20260926 21:23:58.553526 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:58.528887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:48951)
0926 21:23:58.529660 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:58.529729 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:58.532199 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:58.533098 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:58.533107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:58.533626 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:58.534228 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:58.534245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:58.535510 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:58.535518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:58.538265 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:58.538288 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:58.539576 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:58.539585 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:58.539946 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:58.540845 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:58.540881 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:58.544097 (+ 3216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:58.547951 (+ 3854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:58.547964 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:58.547986 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:58.548762 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:58.549439 (+ 677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:58.549445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:58.549449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:58.549715 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:58.550372 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:58.550383 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:58.550877 (+ 494us) client_negotiation.cc:769] Sending connection context
0926 21:23:58.551202 (+ 325us) client_negotiation.cc:240] Negotiation successful
0926 21:23:58.551819 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":323,"threads_started":1}
I20260926 21:23:58.555326 23406 heartbeater.cc:347] Connected to a master server at 127.14.142.62:37799
I20260926 21:23:58.555734 23406 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:58.556852 23406 heartbeater.cc:511] Master 127.14.142.62:37799 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:23:58.559721 23217 ts_manager.cc:194] Registered new tserver with Master: f1c8c5e205c64d5db54fe65ef286d243 (127.14.142.1:44803)
I20260926 21:23:58.561480 23217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:48951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:23:58.592377 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37799
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:58.944070 23412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:58.944366 23412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:58.944463 23412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:58.951792 23412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:58.951915 23412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:58.952148 23412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:23:58.962700 23412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:58.964180 23412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:58.965531 23412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:58.979261 23417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:58.979347 23418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:58.979369 23420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:58.979961 23412 server_base.cc:1034] running on GCE node
I20260926 21:23:58.980731 23412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:58.981721 23412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:58.982977 23412 hybrid_clock.cc:648] HybridClock initialized: now 1790457838982915 us; error 42 us; skew 500 ppm
I20260926 21:23:58.983363 23412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:58 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:58.987282 23412 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:23:58.989043 23412 webserver.cc:460] Webserver started at http://127.14.142.2:44205/ using document root <none> and password file <none>
I20260926 21:23:58.989812 23412 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:58.989964 23412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:58.990321 23412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:58.993755 23412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "c0bcc3ac162044c7856ecfff67c8025d"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "642e077dce587bbeefcb083342862e14"
server_key_iv: "decf97bca8d3ca7cf00c0b22d0120702"
server_key_version: "encryptionkey@0"
I20260926 21:23:58.994748 23412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "c0bcc3ac162044c7856ecfff67c8025d"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "642e077dce587bbeefcb083342862e14"
server_key_iv: "decf97bca8d3ca7cf00c0b22d0120702"
server_key_version: "encryptionkey@0"
I20260926 21:23:59.000633 23412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:23:59.004954 23427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:59.007061 23412 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:23:59.007305 23412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "c0bcc3ac162044c7856ecfff67c8025d"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "642e077dce587bbeefcb083342862e14"
server_key_iv: "decf97bca8d3ca7cf00c0b22d0120702"
server_key_version: "encryptionkey@0"
I20260926 21:23:59.007565 23412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:59.043262 23412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:59.048063 23412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:59.048395 23412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:59.049533 23412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:59.051309 23412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:59.051419 23412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:59.051585 23412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:59.051651 23412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:59.093067 23412 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:43807
I20260926 21:23:59.093305 23539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:43807 every 8 connection(s)
I20260926 21:23:59.094619 23412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:23:59.098999 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23412
I20260926 21:23:59.099196 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:23:59.099627 14904 external_mini_cluster.cc:1428] Setting key 4e042d57e4725194c5e1221968ac043e
Sep 26 21:23:59 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457838, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:59.117596 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:59.097363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43109 (local address 127.14.142.62:37799)
0926 21:23:59.097824 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:59.097828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:59.098578 (+ 750us) server_negotiation.cc:407] Connection header received
0926 21:23:59.100615 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:59.100621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:59.100716 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:59.100836 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:59.102350 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.103079 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.104702 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.105137 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.105268 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:59.110390 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:59.110397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:59.110401 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:59.110443 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:59.113702 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.114848 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.114852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.114854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.115000 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.115604 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.115608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.115610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.115908 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:59.116035 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:59.117138 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0926 21:23:59.117330 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":257,"threads_started":1}
I20260926 21:23:59.119421 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:59.097728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:43109)
0926 21:23:59.098405 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:59.098450 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:59.100315 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:59.101091 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:59.101100 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:59.101602 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:59.102176 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:59.102194 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.103225 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.103231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:59.104491 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:59.104503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.105384 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.105391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:59.105629 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:59.106491 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:59.106533 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:59.110047 (+ 3514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:59.113883 (+ 3836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:59.113897 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:59.113921 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:59.114688 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:59.115130 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:59.115139 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:59.115146 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:59.115446 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:59.116060 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:59.116072 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:59.116652 (+ 580us) client_negotiation.cc:769] Sending connection context
0926 21:23:59.117014 (+ 362us) client_negotiation.cc:240] Negotiation successful
0926 21:23:59.117724 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":268,"threads_started":1}
I20260926 21:23:59.120981 23540 heartbeater.cc:347] Connected to a master server at 127.14.142.62:37799
I20260926 21:23:59.121372 23540 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:59.122406 23540 heartbeater.cc:511] Master 127.14.142.62:37799 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:23:59.124233 23217 ts_manager.cc:194] Registered new tserver with Master: c0bcc3ac162044c7856ecfff67c8025d (127.14.142.2:43807)
I20260926 21:23:59.125144 23217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:43109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:23:59.155560 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37799
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:23:59.505429 23546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:23:59.505748 23546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:23:59.505846 23546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:23:59.513908 23546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:23:59.514046 23546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:23:59.514163 23546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:23:59.526407 23546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:23:59.528000 23546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:23:59.529361 23546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:23:59.542806 23551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:59.542819 23554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:23:59.543141 23552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:23:59.543848 23546 server_base.cc:1034] running on GCE node
I20260926 21:23:59.544538 23546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:23:59.545665 23546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:23:59.546921 23546 hybrid_clock.cc:648] HybridClock initialized: now 1790457839546860 us; error 55 us; skew 500 ppm
I20260926 21:23:59.547264 23546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:23:59 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457839, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:23:59.550585 23546 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:23:59.552209 23546 webserver.cc:460] Webserver started at http://127.14.142.3:42949/ using document root <none> and password file <none>
I20260926 21:23:59.553027 23546 fs_manager.cc:359] Metadata directory not provided
I20260926 21:23:59.553184 23546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:23:59.553519 23546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:23:59.556838 23546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "85941ac2c2c244a38c7d517f6062e4f6"
format_stamp: "Formatted at 2026-09-26 21:23:59 on dist-test-slave-4d5t"
server_key: "b1c1165b5711d5d3ceac78d766b874e4"
server_key_iv: "bd1a43ae50bcd881267edbd45a105df0"
server_key_version: "encryptionkey@0"
I20260926 21:23:59.557739 23546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "85941ac2c2c244a38c7d517f6062e4f6"
format_stamp: "Formatted at 2026-09-26 21:23:59 on dist-test-slave-4d5t"
server_key: "b1c1165b5711d5d3ceac78d766b874e4"
server_key_iv: "bd1a43ae50bcd881267edbd45a105df0"
server_key_version: "encryptionkey@0"
I20260926 21:23:59.563701 23546 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260926 21:23:59.564873 23406 heartbeater.cc:503] Master 127.14.142.62:37799 was elected leader, sending a full tablet report...
I20260926 21:23:59.567787 23561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:59.569491 23546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:23:59.569695 23546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "85941ac2c2c244a38c7d517f6062e4f6"
format_stamp: "Formatted at 2026-09-26 21:23:59 on dist-test-slave-4d5t"
server_key: "b1c1165b5711d5d3ceac78d766b874e4"
server_key_iv: "bd1a43ae50bcd881267edbd45a105df0"
server_key_version: "encryptionkey@0"
I20260926 21:23:59.569917 23546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:23:59.596938 23546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:23:59.601840 23546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:23:59.602169 23546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:23:59.603384 23546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:23:59.605351 23546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:23:59.605499 23546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:59.605681 23546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:23:59.605756 23546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:23:59.645604 23546 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:38395
I20260926 21:23:59.645728 23673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:38395 every 8 connection(s)
I20260926 21:23:59.647066 23546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:23:59.650408 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23546
I20260926 21:23:59.650653 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:23:59.651345 14904 external_mini_cluster.cc:1428] Setting key 9beb3c717d3bfff9e48652fd4c925ece
Sep 26 21:23:59 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457839, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:23:59.672284 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:59.650126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:45631 (local address 127.14.142.62:37799)
0926 21:23:59.651776 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:59.651782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:59.652334 (+ 552us) server_negotiation.cc:407] Connection header received
0926 21:23:59.654778 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:59.654786 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:59.654905 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:59.655017 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:59.656653 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.657513 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.659391 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.659863 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.659983 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:59.664496 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:59.664504 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:59.664508 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:59.664636 (+ 128us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:59.668719 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.669808 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.669814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.669817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.669971 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.670427 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.670432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.670434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.670769 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:59.670916 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:59.671773 (+ 857us) server_negotiation.cc:299] Negotiation successful
0926 21:23:59.671966 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1484,"thread_start_us":1315,"threads_started":1}
I20260926 21:23:59.673414 23677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:59.650633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:45631)
0926 21:23:59.652131 (+ 1498us) negotiation.cc:107] Waiting for socket to connect
0926 21:23:59.652205 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0926 21:23:59.654433 (+ 2228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:23:59.655265 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:23:59.655274 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:23:59.655819 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:23:59.656399 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:59.656417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.657699 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.657704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:59.659112 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:23:59.659126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.660126 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.660132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:23:59.660352 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:59.661134 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:23:59.661158 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:23:59.664070 (+ 2912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:23:59.668957 (+ 4887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:59.668966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:59.668983 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:59.669564 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:59.670093 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:23:59.670097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:23:59.670099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:23:59.670297 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:23:59.670927 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:23:59.670934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:23:59.671349 (+ 415us) client_negotiation.cc:769] Sending connection context
0926 21:23:59.671652 (+ 303us) client_negotiation.cc:240] Negotiation successful
0926 21:23:59.672125 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1230,"thread_start_us":962,"threads_started":1}
I20260926 21:23:59.675674 23674 heartbeater.cc:347] Connected to a master server at 127.14.142.62:37799
I20260926 21:23:59.676057 23674 heartbeater.cc:464] Registering TS with master...
I20260926 21:23:59.676913 23674 heartbeater.cc:511] Master 127.14.142.62:37799 requested a full tablet report, sending...
I20260926 21:23:59.678705 23217 ts_manager.cc:194] Registered new tserver with Master: 85941ac2c2c244a38c7d517f6062e4f6 (127.14.142.3:38395)
I20260926 21:23:59.679697 23217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:45631
I20260926 21:23:59.680866 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab.
Entry for principal oryx/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab.
WARNING: no policy specified for oryx/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab.
Entry for principal oryx/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab.
WARNING: no policy specified for oryx/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab.
Entry for principal oryx/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab.
I20260926 21:23:59.836838 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23278
Sep 26 21:23:59 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:23:59.855496 23685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:59.838190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.14.142.1:44803)
0926 21:23:59.838873 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:59.838883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:59.838915 (+ 32us) server_negotiation.cc:407] Connection header received
0926 21:23:59.838988 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:59.838997 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:59.839277 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:59.839480 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:59.840943 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.842587 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.843915 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.844496 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.844705 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:59.849526 (+ 4821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:59.849549 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:59.849566 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:59.849612 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:59.852757 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.853482 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.853491 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.853493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.853657 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.854055 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.854060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.854062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.854433 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:59.854691 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:59.854982 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 21:23:59.855200 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":340,"threads_started":1}
I20260926 21:23:59.857203 23401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:23:59.941138 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23278
I20260926 21:23:59.957736 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23412
Sep 26 21:23:59 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:23:59.976146 23687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:23:59.958771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56448 (local address 127.14.142.2:43807)
0926 21:23:59.959356 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0926 21:23:59.959363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:23:59.959399 (+ 36us) server_negotiation.cc:407] Connection header received
0926 21:23:59.959466 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:23:59.959473 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:23:59.959675 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:23:59.959805 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:23:59.961106 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.962199 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.963524 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:23:59.964129 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:23:59.964254 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:23:59.969167 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:23:59.969182 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:23:59.969198 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:23:59.969237 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:23:59.973061 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.973818 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.973826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.973828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.973974 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:23:59.974315 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:23:59.974319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:23:59.974321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:23:59.974767 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:23:59.975150 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0926 21:23:59.975451 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 21:23:59.975788 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":330,"threads_started":1}
I20260926 21:23:59.977596 23535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:00.065547 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23412
I20260926 21:24:00.082340 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23546
Sep 26 21:24:00 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:24:00.101056 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:00.083752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.14.142.3:38395)
0926 21:24:00.084513 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:00.084521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:00.084595 (+ 74us) server_negotiation.cc:407] Connection header received
0926 21:24:00.084690 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:00.084696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:00.084928 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:00.085108 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:00.086541 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:00.087546 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:00.088958 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:00.089524 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:00.089673 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:00.094448 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:00.094462 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:00.094478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:00.094521 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:00.097821 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:00.098640 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:00.098654 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:00.098657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:00.098796 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:00.099199 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:00.099206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:00.099210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:00.099607 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:00.099940 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:00.100275 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 21:24:00.100621 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":335,"threads_started":1}
I20260926 21:24:00.102463 23669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:00.192005 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23546
I20260926 21:24:00.209700 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23185
I20260926 21:24:00.211016 23247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:00.317242 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23185
I20260926 21:24:00.339564 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:37799
--webserver_interface=127.14.142.62
--webserver_port=42029
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:37799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:24:00.682013 23692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:00.682330 23692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:00.682487 23692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:00.690148 23692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:24:00.690271 23692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:00.690349 23692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:00.690431 23692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:24:00.690528 23692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:24:00.700994 23692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:37799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:37799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=42029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:00.702442 23692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:00.703855 23692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:00.716617 23697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:00.716681 23698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:00.716645 23700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:00.717355 23692 server_base.cc:1034] running on GCE node
I20260926 21:24:00.718206 23692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:00.719758 23692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:00.721035 23692 hybrid_clock.cc:648] HybridClock initialized: now 1790457840720967 us; error 59 us; skew 500 ppm
I20260926 21:24:00.721462 23692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:00 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457840, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:00.724900 23692 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:24:00.726678 23692 webserver.cc:460] Webserver started at http://127.14.142.62:42029/ using document root <none> and password file <none>
I20260926 21:24:00.727550 23692 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:00.727711 23692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:00.733848 23692 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:24:00.737293 23707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:00.738777 23692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 21:24:00.738982 23692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "e416f3c972bd4f15906edc9782d8978b"
format_stamp: "Formatted at 2026-09-26 21:23:57 on dist-test-slave-4d5t"
server_key: "36551124c462c58630f0b48d69d68743"
server_key_iv: "e7a4db2728ea0f5badbcf8d66ab9a679"
server_key_version: "encryptionkey@0"
I20260926 21:24:00.740079 23692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:00.778002 23692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:00.782912 23692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:00.783252 23692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:00.806345 23692 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:37799
I20260926 21:24:00.806371 23758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:37799 every 8 connection(s)
I20260926 21:24:00.808363 23692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:24:00.815699 23759 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:24:00.818252 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23692
I20260926 21:24:00.819072 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:44803
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:37799
--webserver_port=33709
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:24:00.819073 23759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Bootstrap starting.
I20260926 21:24:00.839149 23759 log.cc:824] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:00.851506 23759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:24:00.852185 23759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Bootstrap complete.
I20260926 21:24:00.866532 23759 raft_consensus.cc:348] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:24:00.867898 23759 raft_consensus.cc:729] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e416f3c972bd4f15906edc9782d8978b, State: Initialized, Role: FOLLOWER
I20260926 21:24:00.868721 23759 consensus_queue.cc:260] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:24:00.869043 23759 raft_consensus.cc:388] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:24:00.869179 23759 raft_consensus.cc:482] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:24:00.869361 23759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 1 FOLLOWER]: Advancing to term 2
I20260926 21:24:00.875644 23759 raft_consensus.cc:504] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:24:00.876335 23759 leader_election.cc:302] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e416f3c972bd4f15906edc9782d8978b; no voters:
I20260926 21:24:00.877082 23759 leader_election.cc:288] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 21:24:00.877322 23763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 2 FOLLOWER]: Leader election won for term 2
I20260926 21:24:00.878700 23763 raft_consensus.cc:686] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [term 2 LEADER]: Becoming Leader. State: Replica: e416f3c972bd4f15906edc9782d8978b, State: Running, Role: LEADER
I20260926 21:24:00.879230 23763 consensus_queue.cc:237] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } }
I20260926 21:24:00.879544 23759 sys_catalog.cc:564] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:24:00.881701 23765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e416f3c972bd4f15906edc9782d8978b. Latest consensus state: current_term: 2 leader_uuid: "e416f3c972bd4f15906edc9782d8978b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } } }
I20260926 21:24:00.882097 23765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:00.882552 23764 sys_catalog.cc:455] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e416f3c972bd4f15906edc9782d8978b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e416f3c972bd4f15906edc9782d8978b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37799 } } }
I20260926 21:24:00.882920 23764 sys_catalog.cc:458] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:00.890631 23768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:24:00.898329 23768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:24:00.900215 23768 catalog_manager.cc:1259] Loaded cluster ID: 3fd4ed6e3f384a59a4e733f0ec0afa38
I20260926 21:24:00.900346 23768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:24:00.905720 23768 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:24:00.907737 23768 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e416f3c972bd4f15906edc9782d8978b: Loaded TSK: 0
I20260926 21:24:00.910073 23768 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 21:24:01.229951 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:01.230302 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:01.230464 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:01.238108 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:01.238271 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:01.238447 23761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:24:01.249125 23761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.1.keytab
--principal=oryx/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:44803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=33709
--tserver_master_addrs=127.14.142.62:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:01.250562 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:01.251886 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:01.267061 23785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:01.266981 23786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:01.267410 23788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:01.269927 23761 server_base.cc:1034] running on GCE node
I20260926 21:24:01.270670 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:01.272287 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:01.273568 23761 hybrid_clock.cc:648] HybridClock initialized: now 1790457841273494 us; error 68 us; skew 500 ppm
I20260926 21:24:01.273989 23761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:01 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457841, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:01.278471 23761 init.cc:377] Logged in from keytab as oryx/127.14.142.1@KRBTEST.COM (short username oryx)
I20260926 21:24:01.280786 23761 webserver.cc:460] Webserver started at http://127.14.142.1:33709/ using document root <none> and password file <none>
I20260926 21:24:01.282064 23761 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:01.282279 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:01.290242 23761 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 21:24:01.293656 23795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:01.295680 23761 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:24:01.295965 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "f1c8c5e205c64d5db54fe65ef286d243"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "2db0e1a230258698a710ab57171f1304"
server_key_iv: "22e6841c211ce6fd90616f9915e50523"
server_key_version: "encryptionkey@0"
I20260926 21:24:01.297088 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:01.313136 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:01.317552 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:01.317845 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:01.318931 23761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:01.320343 23761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:01.320467 23761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:01.320621 23761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:01.320716 23761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:01.359047 23761 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:44803
I20260926 21:24:01.359117 23907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:44803 every 8 connection(s)
I20260926 21:24:01.360738 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:24:01.367821 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23761
I20260926 21:24:01.368556 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:43807
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:37799
--webserver_port=44205
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:24:01 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:01 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 21:24:01.734807 23912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:01.735270 23912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:01.735447 23912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:01.744061 23912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:01.744261 23912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:01.744500 23912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:24:01.755867 23912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.2.keytab
--principal=oryx/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:43807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=44205
--tserver_master_addrs=127.14.142.62:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:01.757642 23912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:01.759050 23912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:01.773927 23919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:01.774031 23918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:01.774891 23921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:01.775734 23912 server_base.cc:1034] running on GCE node
I20260926 21:24:01.777096 23912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:01.778957 23912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:01.780287 23912 hybrid_clock.cc:648] HybridClock initialized: now 1790457841780184 us; error 78 us; skew 500 ppm
I20260926 21:24:01.781019 23912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:01 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457841, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:01.787278 23912 init.cc:377] Logged in from keytab as oryx/127.14.142.2@KRBTEST.COM (short username oryx)
I20260926 21:24:01.790150 23912 webserver.cc:460] Webserver started at http://127.14.142.2:44205/ using document root <none> and password file <none>
I20260926 21:24:01.791247 23912 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:01.791414 23912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:01.801065 23912 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.004s
I20260926 21:24:01.805641 23928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:01.808495 23912 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260926 21:24:01.808779 23912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "c0bcc3ac162044c7856ecfff67c8025d"
format_stamp: "Formatted at 2026-09-26 21:23:58 on dist-test-slave-4d5t"
server_key: "642e077dce587bbeefcb083342862e14"
server_key_iv: "decf97bca8d3ca7cf00c0b22d0120702"
server_key_version: "encryptionkey@0"
I20260926 21:24:01.810374 23912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:01.833314 23912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:01.837769 23912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:01.838063 23912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:01.839236 23912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:01.840894 23912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:01.841001 23912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:01.841146 23912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:01.841221 23912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:01.885834 23912 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:43807
I20260926 21:24:01.885881 24040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:43807 every 8 connection(s)
I20260926 21:24:01.887302 23912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:24:01.891242 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 23912
I20260926 21:24:01.891839 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:38395
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:37799
--webserver_port=42949
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:24:01 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:01 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 21:24:02.584501 23911 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.206s user 0.007s sys 0.011s
I20260926 21:24:02.586665 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:01.363785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:52745)
0926 21:24:01.364735 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:01.364779 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:01.366505 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:01.368706 (+ 2201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:01.368721 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:01.369569 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:01.370465 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:01.370493 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.372925 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.372934 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:01.375192 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:01.375214 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.377189 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.377195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:01.377429 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:01.378162 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:01.378191 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:02.585416 (+1207225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":467,"threads_started":1}
W20260926 21:24:02.586670 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:01.363769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:52745 (local address 127.14.142.62:37799)
0926 21:24:01.364465 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:01.364476 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:01.364949 (+ 473us) server_negotiation.cc:407] Connection header received
0926 21:24:01.367343 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:01.367370 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:01.367878 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:01.368380 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:01.370803 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.372714 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.375436 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.376888 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.377118 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.585906 (+1208788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:52745: BlockingRecv error: recv got EOF from 127.14.142.1:52745 (error 108)
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":294,"threads_started":1}
W20260926 21:24:02.587939 23908 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:37799 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.142.62:37799: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 21:24:02.598201 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:02.588842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:43037 (local address 127.14.142.62:37799)
0926 21:24:02.589017 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:02.589024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:02.589135 (+ 111us) server_negotiation.cc:407] Connection header received
0926 21:24:02.589343 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:02.589353 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:02.589414 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:02.589517 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:02.590251 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.590989 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.592303 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.592847 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.593049 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.597968 (+ 4919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:43037: BlockingRecv error: recv got EOF from 127.14.142.1:43037 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 21:24:02.598203 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:02.588763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:43037)
0926 21:24:02.589026 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:02.589046 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:02.589212 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:02.589546 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:02.589551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:02.589948 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:02.590111 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:02.590120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.591139 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.591146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:02.592163 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:02.592174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.593080 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.593086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:02.593265 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.593950 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:02.594013 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:02.597741 (+ 3728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:02.608451 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:02.599205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:51675)
0926 21:24:02.599462 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:02.599481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:02.599599 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:02.599948 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:02.599952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:02.600295 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:02.600540 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:02.600586 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.601811 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.601817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:02.602890 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:02.602903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.603638 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.603645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:02.603842 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.604492 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:02.604510 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:02.608041 (+ 3531us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148}
W20260926 21:24:02.608516 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:02.599329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51675 (local address 127.14.142.62:37799)
0926 21:24:02.599479 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:02.599485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:02.599534 (+ 49us) server_negotiation.cc:407] Connection header received
0926 21:24:02.599731 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:02.599736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:02.599811 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:02.599914 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:02.600729 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.601669 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.603033 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.603459 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.603587 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.608290 (+ 4703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:51675: BlockingRecv error: recv got EOF from 127.14.142.1:51675 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 21:24:02.609108 23908 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 21:24:02.663156 24045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:02.663789 24045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:02.664045 24045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:02.683368 24045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:02.683940 24045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:02.684443 24045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:24:02.714349 24045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/krb5kdc/oryx_127.14.142.3.keytab
--principal=oryx/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:38395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=42949
--tserver_master_addrs=127.14.142.62:37799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:02.717244 24045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:02.720034 24045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:02.741966 24052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:02.744387 24045 server_base.cc:1034] running on GCE node
W20260926 21:24:02.744520 24051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:02.745186 24054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:02.746522 24045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:02.748402 24045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:02.749750 24045 hybrid_clock.cc:648] HybridClock initialized: now 1790457842749680 us; error 66 us; skew 500 ppm
I20260926 21:24:02.750376 24045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457842, etypes {rep=17 tkt=17 ses=17}, oryx/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:02.756491 24045 init.cc:377] Logged in from keytab as oryx/127.14.142.3@KRBTEST.COM (short username oryx)
I20260926 21:24:02.759466 24045 webserver.cc:460] Webserver started at http://127.14.142.3:42949/ using document root <none> and password file <none>
I20260926 21:24:02.760900 24045 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:02.761170 24045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:02.773156 24045 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 21:24:02.778105 24061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:02.780510 24045 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260926 21:24:02.780998 24045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "85941ac2c2c244a38c7d517f6062e4f6"
format_stamp: "Formatted at 2026-09-26 21:23:59 on dist-test-slave-4d5t"
server_key: "b1c1165b5711d5d3ceac78d766b874e4"
server_key_iv: "bd1a43ae50bcd881267edbd45a105df0"
server_key_version: "encryptionkey@0"
I20260926 21:24:02.783144 24045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:02.815107 24045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:02.823359 24045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:02.823837 24045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:02.825644 24045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:02.828291 24045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:02.828476 24045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:02.828681 24045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:02.828799 24045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:02.871501 24045 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:38395
I20260926 21:24:02.871496 24173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:38395 every 8 connection(s)
I20260926 21:24:02.874177 24045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:24:02.884456 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24045
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:02 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:02.915688 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:02.886202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.14.142.62:37799)
0926 21:24:02.894563 (+ 8361us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:02.894573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:02.899118 (+ 4545us) server_negotiation.cc:407] Connection header received
0926 21:24:02.899364 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:02.899371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:02.899466 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:02.899594 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:02.901445 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.902542 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.903957 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.904407 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.904557 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.907532 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:02.907541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:02.907561 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:02.907610 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:02.911879 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:02.912736 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:02.912744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:02.912753 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:02.912951 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:02.913352 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:02.913355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:02.913357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:02.913882 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:02.914228 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:02.914694 (+ 466us) server_negotiation.cc:299] Negotiation successful
0926 21:24:02.915393 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8141,"thread_start_us":8449,"threads_started":1}
W20260926 21:24:03.127614 24044 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.222s user 0.003s sys 0.012s
W20260926 21:24:03.129022 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:01.890390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:52573 (local address 127.14.142.62:37799)
0926 21:24:01.890838 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:01.890845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:01.893462 (+ 2617us) server_negotiation.cc:407] Connection header received
0926 21:24:01.896731 (+ 3269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:01.896742 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:01.896890 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:01.897069 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:01.899536 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.901448 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.903465 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.903895 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.904047 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.128674 (+1224627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:52573: BlockingRecv error: recv got EOF from 127.14.142.2:52573 (error 108)
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":254,"threads_started":1}
I20260926 21:24:03.129508 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:01.891625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:52573)
0926 21:24:01.893255 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:01.893314 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:01.896230 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:01.897383 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:01.897399 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:01.898283 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:01.899211 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:01.899241 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.901682 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.901687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:01.903261 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:01.903278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:01.904153 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:01.904163 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:01.904444 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:01.905202 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:01.905226 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:03.128401 (+1223175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1231,"thread_start_us":954,"threads_started":1}
W20260926 21:24:03.130319 24041 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:37799 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.142.62:37799: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Sep 26 21:24:03 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:03 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:03.144840 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:03.130969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:41103)
0926 21:24:03.131293 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:03.131323 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:03.131517 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:03.131922 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:03.131930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:03.132670 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:03.132982 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:03.132999 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.134198 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.134206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:03.136651 (+ 2445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:03.136675 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.137488 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.137498 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:03.137765 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.138596 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:03.138630 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:03.144247 (+ 5617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260926 21:24:03.145051 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:03.131102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:41103 (local address 127.14.142.62:37799)
0926 21:24:03.131252 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:03.131256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:03.131441 (+ 185us) server_negotiation.cc:407] Connection header received
0926 21:24:03.131672 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:03.131678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:03.131761 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:03.131864 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:03.133187 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.133979 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.136831 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.137262 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.137359 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.144754 (+ 7395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:41103: BlockingRecv error: recv got EOF from 127.14.142.2:41103 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 21:24:03 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:03 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:03.158988 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:03.145792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:57229)
0926 21:24:03.145987 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:03.146011 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:03.146132 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:03.147166 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:03.147172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:03.147705 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:03.147996 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:03.148015 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.150330 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.150336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:03.151642 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:03.151658 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.153167 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.153173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:03.153383 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.154334 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:03.154356 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:03.158768 (+ 4412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 21:24:03.159363 24041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 21:24:03.159626 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:03.146157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:57229 (local address 127.14.142.62:37799)
0926 21:24:03.146777 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:03.146784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:03.146808 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:24:03.146884 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:03.146891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:03.146990 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:03.147120 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:03.148193 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.150017 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.151820 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.152898 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.153095 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.159221 (+ 6126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:57229: BlockingRecv error: recv got EOF from 127.14.142.2:57229 (error 108)
Metrics: {"server-negotiator.queue_time_us":457}
Sep 26 21:24:03 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:03 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:03.622411 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:03.609963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:36987)
0926 21:24:03.610621 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:03.610642 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:03.610821 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:03.611381 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:03.611386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:03.611872 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:03.612048 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:03.612059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.614235 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.614243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:03.615241 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:03.615253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.616682 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.616695 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:03.616863 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.617665 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:03.617692 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:03.622012 (+ 4320us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":546,"threads_started":1}
W20260926 21:24:03.622836 24043 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:03.610019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:36987 (local address 127.14.142.62:37799)
0926 21:24:03.610204 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:03.610210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:03.610713 (+ 503us) server_negotiation.cc:407] Connection header received
0926 21:24:03.611005 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:03.611015 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:03.611183 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:03.611333 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:03.612211 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.613893 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.615402 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:03.616354 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:03.616524 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:03.622247 (+ 5723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:36987: BlockingRecv error: recv got EOF from 127.14.142.1:36987 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 21:24:04.023439 24176 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.132s user 0.008s sys 0.018s
W20260926 21:24:04.024781 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:02.877632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:37133 (local address 127.14.142.62:37799)
0926 21:24:02.877807 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:02.877812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:02.879733 (+ 1921us) server_negotiation.cc:407] Connection header received
0926 21:24:02.881731 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:02.881737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:02.881830 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:02.881941 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:02.883544 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.884439 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.888132 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.889319 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.889473 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.024465 (+1134992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:37133: BlockingRecv error: recv got EOF from 127.14.142.3:37133 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 21:24:04.025331 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:02.878406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:37133)
0926 21:24:02.879368 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:02.879593 (+ 225us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:02.881428 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:02.882189 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:02.882197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:02.882720 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:02.883360 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:02.883376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.884713 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.884721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:02.887813 (+ 3092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:02.887840 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:02.890029 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:02.890041 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:02.890615 (+ 574us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:02.891736 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:02.891796 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:04.024263 (+1132467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":426,"threads_started":1}
W20260926 21:24:04.026257 24174 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:37799 (0 consecutive failures): Not authorized: Failed to ping master at 127.14.142.62:37799: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:04.036865 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:04.026869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:49239)
0926 21:24:04.027087 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:04.027109 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:04.027229 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:04.027539 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:04.027544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:04.027925 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:04.028094 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.028102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.029227 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.029232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.030306 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.030318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.031400 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.031412 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.031639 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.032468 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:04.032491 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:04.036500 (+ 4009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260926 21:24:04.037060 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:04.026995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:49239 (local address 127.14.142.62:37799)
0926 21:24:04.027224 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:04.027230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:04.027250 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:24:04.027340 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:04.027349 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:04.027410 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:04.027519 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:04.028229 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.029090 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.030434 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.030972 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.031104 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.036769 (+ 5665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:49239: BlockingRecv error: recv got EOF from 127.14.142.3:49239 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:04.047421 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:04.037662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:47941)
0926 21:24:04.037853 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:04.037873 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:04.037972 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:04.038265 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:04.038268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:04.038581 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:04.038763 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.038772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.039866 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.039870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.041140 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.041152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.041845 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.041850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.041996 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.042680 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:04.042700 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:04.047189 (+ 4489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 21:24:04.047659 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:04.037778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:47941 (local address 127.14.142.62:37799)
0926 21:24:04.037954 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:04.037959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:04.037977 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:24:04.038061 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:04.038065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:04.038132 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:04.038218 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:04.038940 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.039725 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.041279 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.041699 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.041797 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.047435 (+ 5638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:47941: BlockingRecv error: recv got EOF from 127.14.142.3:47941 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260926 21:24:04.047864 24174 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:04.169580 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:04.159914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:42007)
0926 21:24:04.160489 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:04.160505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:04.160677 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:04.160988 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:04.160991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:04.161395 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:04.161537 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.161545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.162563 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.162567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.163867 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.163891 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.164788 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.164793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.164913 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.165522 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:04.165539 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:04.169158 (+ 3619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":354,"threads_started":1}
W20260926 21:24:04.169692 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:04.160083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:42007 (local address 127.14.142.62:37799)
0926 21:24:04.160285 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:04.160290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:04.160601 (+ 311us) server_negotiation.cc:407] Connection header received
0926 21:24:04.160777 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:04.160781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:04.160863 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:04.160957 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:04.161681 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.162417 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.164061 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.164506 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.164728 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.169400 (+ 4672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:42007: BlockingRecv error: recv got EOF from 127.14.142.2:42007 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:04 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:04.634202 24182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:04.623690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:38721)
0926 21:24:04.624279 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:04.624296 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:04.624425 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:04.624986 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:04.624990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:04.625400 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:04.625605 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.625615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.626882 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.626887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.627871 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:04.627883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.629155 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.629161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:04.629341 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.630051 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:04.630102 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:04.633811 (+ 3709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":344,"threads_started":1}
W20260926 21:24:04.634267 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:04.623832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:38721 (local address 127.14.142.62:37799)
0926 21:24:04.624045 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:04.624050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:04.624473 (+ 423us) server_negotiation.cc:407] Connection header received
0926 21:24:04.624634 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:04.624638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:04.624746 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:04.624881 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:04.625844 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.626701 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.628069 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:04.628953 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:04.629110 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:04.634032 (+ 4922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:38721: BlockingRecv error: recv got EOF from 127.14.142.1:38721 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 26 21:24:05 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:05 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:05.061698 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:05.048621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:42117)
0926 21:24:05.049536 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:05.049556 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:05.049751 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:05.050140 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:05.050144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:05.050972 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:05.051153 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:05.051162 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.052406 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.052414 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:05.054390 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:05.054409 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.055710 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.055718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:05.055887 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:05.056747 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:05.056772 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:05.061381 (+ 4609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":563,"threads_started":1}
W20260926 21:24:05.061962 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:05.048731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:42117 (local address 127.14.142.62:37799)
0926 21:24:05.049227 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:05.049237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:05.049689 (+ 452us) server_negotiation.cc:407] Connection header received
0926 21:24:05.049898 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:05.049902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:05.050005 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:05.050114 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:05.051383 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.052239 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.054585 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.055267 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.055421 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:05.061676 (+ 6255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:42117: BlockingRecv error: recv got EOF from 127.14.142.3:42117 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
Sep 26 21:24:05 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:05 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:05.181519 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:05.171000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:56557)
0926 21:24:05.171651 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:05.171675 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:05.171899 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:05.172285 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:05.172290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:05.172911 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:05.173127 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:05.173136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.174540 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.174548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:05.175524 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:05.175538 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.176299 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.176305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:05.176489 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:05.177265 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:05.177284 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:05.181256 (+ 3972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":372,"threads_started":1}
W20260926 21:24:05.181810 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:05.171252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:56557 (local address 127.14.142.62:37799)
0926 21:24:05.171492 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:05.171499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:05.171795 (+ 296us) server_negotiation.cc:407] Connection header received
0926 21:24:05.172038 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:05.172044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:05.172125 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:05.172253 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:05.173314 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.174319 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.175703 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.176134 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.176279 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:05.181526 (+ 5247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:56557: BlockingRecv error: recv got EOF from 127.14.142.2:56557 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 21:24:05 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:05 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 21:24:05.646106 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:05.635657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:35501 (local address 127.14.142.62:37799)
0926 21:24:05.635849 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:05.635853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:05.636226 (+ 373us) server_negotiation.cc:407] Connection header received
0926 21:24:05.636324 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:05.636329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:05.636409 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:05.636521 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:05.637865 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.638772 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.640240 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.640700 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.640797 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:05.645837 (+ 5040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:35501: BlockingRecv error: recv got EOF from 127.14.142.1:35501 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 21:24:05.646188 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:05.635474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:35501)
0926 21:24:05.636085 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:05.636102 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:05.636224 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:05.636775 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:05.636781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:05.637485 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:05.637697 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:05.637710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.638927 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.638932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:05.640052 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:05.640089 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:05.640841 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:05.640845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:05.640976 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:05.641722 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:05.641741 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:05.645571 (+ 3830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":344,"threads_started":1}
Sep 26 21:24:06 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:06 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:06.073360 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:06.063059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:58597)
0926 21:24:06.063728 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:06.063748 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:06.063892 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:06.064409 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:06.064414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:06.064922 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:06.065129 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:06.065139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.066296 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.066301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:06.067277 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:06.067291 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.068071 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.068075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:06.068201 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:06.069027 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:06.069052 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:06.073064 (+ 4012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":384,"threads_started":1}
W20260926 21:24:06.073530 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:06.063312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:58597 (local address 127.14.142.62:37799)
0926 21:24:06.063492 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:06.063496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:06.063886 (+ 390us) server_negotiation.cc:407] Connection header received
0926 21:24:06.064003 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:06.064008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:06.064142 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:06.064276 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:06.065320 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.066119 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.067435 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.067922 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.068030 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:06.073285 (+ 5255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:58597: BlockingRecv error: recv got EOF from 127.14.142.3:58597 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 26 21:24:06 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:06 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:06.193780 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:06.182696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:59597)
0926 21:24:06.183429 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:06.183449 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:06.183661 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:06.184085 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:06.184090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:06.184865 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:06.185106 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:06.185116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.186630 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.186642 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:06.187619 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:06.187636 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.188411 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.188418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:06.188639 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:06.189392 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:06.189414 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:06.193415 (+ 4001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":393,"threads_started":1}
W20260926 21:24:06.194042 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:06.182920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:59597 (local address 127.14.142.62:37799)
0926 21:24:06.183219 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:06.183226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:06.183563 (+ 337us) server_negotiation.cc:407] Connection header received
0926 21:24:06.183841 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:06.183847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:06.183935 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:06.184045 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:06.185277 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.186410 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.187811 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.188248 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.188392 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:06.193719 (+ 5327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:59597: BlockingRecv error: recv got EOF from 127.14.142.2:59597 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Sep 26 21:24:06 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:06 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:06.658349 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:06.647396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:36789)
0926 21:24:06.648121 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:06.648139 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:06.648314 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:06.648723 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:06.648728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:06.649206 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:06.649383 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:06.649393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.650484 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.650493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:06.651617 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:06.651630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.652405 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.652412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:06.652685 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:06.653401 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:06.653422 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:06.657931 (+ 4509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":491,"threads_started":1}
W20260926 21:24:06.658420 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:06.647546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:36789 (local address 127.14.142.62:37799)
0926 21:24:06.647772 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:06.647782 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:06.648247 (+ 465us) server_negotiation.cc:407] Connection header received
0926 21:24:06.648461 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:06.648467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:06.648546 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:06.648687 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:06.649547 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.650341 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.651782 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:06.652235 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:06.652374 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:06.658178 (+ 5804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:36789: BlockingRecv error: recv got EOF from 127.14.142.1:36789 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 21:24:07 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:07 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:07.086696 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:07.074680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:48115)
0926 21:24:07.075384 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:07.075405 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:07.075533 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:07.075953 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:07.075958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:07.076760 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:07.076937 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:07.076947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.078437 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.078443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:07.079524 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:07.079543 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.080793 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.080801 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:07.081071 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:07.081842 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:07.081866 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:07.086294 (+ 4428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":390,"threads_started":1}
W20260926 21:24:07.086890 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:07.074672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48115 (local address 127.14.142.62:37799)
0926 21:24:07.075006 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:07.075015 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:07.075526 (+ 511us) server_negotiation.cc:407] Connection header received
0926 21:24:07.075632 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:07.075636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:07.075725 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:07.075824 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:07.077131 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.078216 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.079879 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.080491 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.080752 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:07.086534 (+ 5782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:48115: BlockingRecv error: recv got EOF from 127.14.142.3:48115 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 26 21:24:07 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:07 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:07.207543 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:07.195200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:43159)
0926 21:24:07.195946 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:07.196000 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:07.196213 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:07.196666 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:07.196672 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:07.197193 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:07.197396 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:07.197405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.198540 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.198547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:07.199966 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:07.199993 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.201271 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.201276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:07.201433 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:07.202422 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:07.202448 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:07.207097 (+ 4649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":396,"threads_started":1}
W20260926 21:24:07.207571 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:07.195365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43159 (local address 127.14.142.62:37799)
0926 21:24:07.195692 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:07.195700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:07.196104 (+ 404us) server_negotiation.cc:407] Connection header received
0926 21:24:07.196386 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:07.196390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:07.196473 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:07.196615 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:07.197550 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.198389 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.200241 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.200969 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.201167 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:07.207339 (+ 6172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:43159: BlockingRecv error: recv got EOF from 127.14.142.2:43159 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Sep 26 21:24:07 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:07 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:07.671201 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:07.659597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:48411)
0926 21:24:07.660379 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:07.660401 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:07.660736 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:07.661255 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:07.661260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:07.661838 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:07.662025 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:07.662034 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.663203 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.663210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:07.664302 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:07.664314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.665208 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.665214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:07.665429 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:07.666236 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:07.666263 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:07.670844 (+ 4581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":421,"threads_started":1}
W20260926 21:24:07.671324 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:07.659908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:48411 (local address 127.14.142.62:37799)
0926 21:24:07.660255 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:07.660264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:07.660529 (+ 265us) server_negotiation.cc:407] Connection header received
0926 21:24:07.661003 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:07.661008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:07.661093 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:07.661215 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:07.662197 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.663043 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.664474 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:07.665032 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:07.665189 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:07.671067 (+ 5878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:48411: BlockingRecv error: recv got EOF from 127.14.142.1:48411 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Sep 26 21:24:08 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:08 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:08.105265 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:08.088285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:48423)
0926 21:24:08.088990 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:08.089012 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:08.089193 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:08.089673 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:08.089677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:08.090175 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:08.090365 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:08.090375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.091771 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.091776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:08.093453 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:08.093492 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.094421 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.094427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:08.094615 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:08.100473 (+ 5858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:08.100499 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:08.104843 (+ 4344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":408,"threads_started":1}
W20260926 21:24:08.105433 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:08.088399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48423 (local address 127.14.142.62:37799)
0926 21:24:08.088743 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:08.088751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:08.089153 (+ 402us) server_negotiation.cc:407] Connection header received
0926 21:24:08.089376 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:08.089381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:08.089476 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:08.089596 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:08.090540 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.091598 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.093644 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.094257 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.094407 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:08.105146 (+ 10739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:48423: BlockingRecv error: recv got EOF from 127.14.142.3:48423 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Sep 26 21:24:08 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:08 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:08.219431 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:08.208542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:44241)
0926 21:24:08.209074 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:08.209093 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:08.209268 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:08.209619 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:08.209634 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:08.210101 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:08.210260 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:08.210269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.211497 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.211505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:08.212787 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:08.212802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.213584 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.213590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:08.213780 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:08.214455 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:08.214472 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:08.219117 (+ 4645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":377,"threads_started":1}
W20260926 21:24:08.219859 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:08.208750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:44241 (local address 127.14.142.62:37799)
0926 21:24:08.208942 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:08.208950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:08.209194 (+ 244us) server_negotiation.cc:407] Connection header received
0926 21:24:08.209389 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:08.209395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:08.209468 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:08.209570 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:08.210413 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.211159 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.212962 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.213412 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.213567 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:08.219544 (+ 5977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:44241: BlockingRecv error: recv got EOF from 127.14.142.2:44241 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 26 21:24:08 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:08 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:08.681805 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:08.672245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:57101)
0926 21:24:08.672778 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:08.672794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:08.672908 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:08.673290 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:08.673293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:08.673701 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:08.673865 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:08.673873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.674970 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.674973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:08.675929 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:08.675939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.676731 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.676736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:08.676868 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:08.677548 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:08.677568 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:08.681443 (+ 3875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":304,"threads_started":1}
W20260926 21:24:08.681914 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:08.672312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:57101 (local address 127.14.142.62:37799)
0926 21:24:08.672526 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:08.672530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:08.672900 (+ 370us) server_negotiation.cc:407] Connection header received
0926 21:24:08.673030 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:08.673034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:08.673134 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:08.673241 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:08.674044 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.674823 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.676082 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:08.676577 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:08.676690 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:08.681680 (+ 4990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:57101: BlockingRecv error: recv got EOF from 127.14.142.1:57101 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Sep 26 21:24:09 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:09 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:09.116825 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:09.106524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:49883)
0926 21:24:09.107078 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:09.107094 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:09.107254 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:09.107588 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:09.107592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:09.108056 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:09.108208 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:09.108217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.109323 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.109329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:09.110357 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:09.110368 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.111113 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.111119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:09.111298 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:09.112035 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:09.112055 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:09.116459 (+ 4404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":307,"threads_started":1}
W20260926 21:24:09.117026 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:09.106651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:49883 (local address 127.14.142.62:37799)
0926 21:24:09.106892 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:09.106901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:09.107186 (+ 285us) server_negotiation.cc:407] Connection header received
0926 21:24:09.107381 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:09.107391 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:09.107451 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:09.107557 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:09.108354 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.109179 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.110491 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.110959 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.111092 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:09.116757 (+ 5665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:49883: BlockingRecv error: recv got EOF from 127.14.142.3:49883 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 21:24:09 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:09 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:09.232106 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:09.220983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:41617)
0926 21:24:09.221849 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:09.221877 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:09.222064 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:09.222490 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:09.222494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:09.223068 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:09.223309 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:09.223322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.224688 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.224693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:09.225731 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:09.225744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.226549 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.226555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:09.226741 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:09.227597 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:09.227620 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:09.231848 (+ 4228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":480,"threads_started":1}
W20260926 21:24:09.232453 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:09.221135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:41617 (local address 127.14.142.62:37799)
0926 21:24:09.221440 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:09.221445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:09.222026 (+ 581us) server_negotiation.cc:407] Connection header received
0926 21:24:09.222230 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:09.222235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:09.222317 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:09.222440 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:09.223495 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.224483 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.225893 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.226323 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.226434 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:09.232148 (+ 5714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:41617: BlockingRecv error: recv got EOF from 127.14.142.2:41617 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Sep 26 21:24:09 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:09 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:09.692634 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:09.683088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:41117)
0926 21:24:09.683731 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:09.683747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:09.683947 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:09.684326 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:09.684331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:09.684912 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:09.685085 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:09.685094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.686196 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.686204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:09.687150 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:09.687161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.687907 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.687913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:09.688085 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:09.688776 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:09.688794 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:09.692245 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":310,"threads_started":1}
W20260926 21:24:09.692775 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:09.683195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:41117 (local address 127.14.142.62:37799)
0926 21:24:09.683398 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:09.683407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:09.683844 (+ 437us) server_negotiation.cc:407] Connection header received
0926 21:24:09.684105 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:09.684111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:09.684176 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:09.684301 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:09.685234 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.686038 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.687286 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:09.687750 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:09.687879 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:09.692469 (+ 4590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:41117: BlockingRecv error: recv got EOF from 127.14.142.1:41117 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 21:24:10 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:10 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:10.129078 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:10.117957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:46061)
0926 21:24:10.118679 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:10.118704 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:10.118821 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:10.119203 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:10.119207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:10.119603 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:10.119787 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:10.119796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.121497 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.121502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:10.122443 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:10.122455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.123597 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.123602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:10.123745 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:10.124456 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:10.124476 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:10.128828 (+ 4352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":491,"threads_started":1}
W20260926 21:24:10.129595 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:10.118057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:46061 (local address 127.14.142.62:37799)
0926 21:24:10.118288 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:10.118298 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:10.118794 (+ 496us) server_negotiation.cc:407] Connection header received
0926 21:24:10.118935 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:10.118942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:10.119038 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:10.119155 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:10.119937 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.121311 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.122600 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.123414 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.123577 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:10.129129 (+ 5552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:46061: BlockingRecv error: recv got EOF from 127.14.142.3:46061 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 26 21:24:10 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:10 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:10.246374 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:10.233334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:42567)
0926 21:24:10.234093 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:10.234112 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:10.234257 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:10.234752 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:10.234756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:10.235194 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:10.235375 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:10.235383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.237244 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.237259 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:10.238327 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:10.238341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.240021 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.240026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:10.240164 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:10.240917 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:10.240945 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:10.246035 (+ 5090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":357,"threads_started":1}
W20260926 21:24:10.246914 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:10.233393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:42567 (local address 127.14.142.62:37799)
0926 21:24:10.233710 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:10.233719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:10.234219 (+ 500us) server_negotiation.cc:407] Connection header received
0926 21:24:10.234409 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:10.234417 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:10.234523 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:10.234641 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:10.235532 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.236917 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.238502 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.239814 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.239987 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:10.246370 (+ 6383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:42567: BlockingRecv error: recv got EOF from 127.14.142.2:42567 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Sep 26 21:24:10 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:10 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:10.704753 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:10.693806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:42749)
0926 21:24:10.694398 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:10.694415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:10.694591 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:10.694919 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:10.694923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:10.695373 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:10.695547 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:10.695555 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.696595 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.696601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:10.697535 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:10.697545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.698279 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.698284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:10.698457 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:10.699351 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:10.699384 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:10.704405 (+ 5021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":275,"threads_started":1}
W20260926 21:24:10.704990 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:10.693832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:42749 (local address 127.14.142.62:37799)
0926 21:24:10.694041 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:10.694050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:10.694513 (+ 463us) server_negotiation.cc:407] Connection header received
0926 21:24:10.694736 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:10.694740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:10.694800 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:10.694897 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:10.695690 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.696420 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.697675 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:10.698096 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:10.698244 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:10.704753 (+ 6509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:42749: BlockingRecv error: recv got EOF from 127.14.142.1:42749 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 26 21:24:11 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:11 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:11.140781 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:11.130140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:43331)
0926 21:24:11.130816 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:11.130836 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:11.130978 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:11.131427 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:11.131431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:11.132053 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:11.132231 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:11.132240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.133770 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.133774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:11.134738 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:11.134749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.135681 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.135686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:11.135796 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:11.136444 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:11.136462 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:11.140426 (+ 3964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":347,"threads_started":1}
W20260926 21:24:11.140990 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:11.130455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:43331 (local address 127.14.142.62:37799)
0926 21:24:11.130678 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:11.130684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:11.130933 (+ 249us) server_negotiation.cc:407] Connection header received
0926 21:24:11.131091 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:11.131097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:11.131217 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:11.131333 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:11.132479 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.133615 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.134913 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.135518 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.135637 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:11.140701 (+ 5064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:43331: BlockingRecv error: recv got EOF from 127.14.142.3:43331 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 26 21:24:11 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:11 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:11.258417 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:11.247698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:52653)
0926 21:24:11.248492 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:11.248510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:11.248735 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:11.249081 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:11.249086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:11.249527 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:11.249694 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:11.249703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.250857 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.250864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:11.251825 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:11.251837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.252608 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.252614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:11.252776 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:11.253441 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:11.253459 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:11.257822 (+ 4363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":486,"threads_started":1}
W20260926 21:24:11.258615 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:11.247865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:52653 (local address 127.14.142.62:37799)
0926 21:24:11.248263 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:11.248271 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:11.248619 (+ 348us) server_negotiation.cc:407] Connection header received
0926 21:24:11.248871 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:11.248878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:11.248945 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:11.249051 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:11.249864 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.250718 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.251972 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.252370 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.252504 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:11.258130 (+ 5626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:52653: BlockingRecv error: recv got EOF from 127.14.142.2:52653 (error 108)
Metrics: {"server-negotiator.queue_time_us":206}
Sep 26 21:24:11 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:11 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:11.715941 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:11.705958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:50559)
0926 21:24:11.706439 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:11.706456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:11.706569 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:11.706877 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:11.706880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:11.707297 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:11.707448 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:11.707457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.708717 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.708722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:11.709878 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:11.709891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.710624 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.710628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:11.710755 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:11.711360 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:11.711379 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:11.715607 (+ 4228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":261,"threads_started":1}
W20260926 21:24:11.716109 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:11.705989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:50559 (local address 127.14.142.62:37799)
0926 21:24:11.706224 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:11.706234 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:11.706560 (+ 326us) server_negotiation.cc:407] Connection header received
0926 21:24:11.706658 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:11.706662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:11.706744 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:11.706840 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:11.707650 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.708491 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.710031 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:11.710471 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:11.710575 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:11.715852 (+ 5277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:50559: BlockingRecv error: recv got EOF from 127.14.142.1:50559 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 21:24:12 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:12 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:12.151674 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:12.141947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:44035)
0926 21:24:12.142569 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:12.142598 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:12.142757 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:12.143094 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:12.143099 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:12.143635 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:12.143831 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:12.143840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.145034 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.145041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:12.146107 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:12.146119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.146906 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.146911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:12.147089 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:12.147722 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:12.147740 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:12.151297 (+ 3557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":333,"threads_started":1}
W20260926 21:24:12.151762 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:12.142052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:44035 (local address 127.14.142.62:37799)
0926 21:24:12.142271 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:12.142284 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:12.142738 (+ 454us) server_negotiation.cc:407] Connection header received
0926 21:24:12.142894 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:12.142899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:12.142961 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:12.143066 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:12.143974 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.144889 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.146251 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.146709 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.146849 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:12.151530 (+ 4681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:44035: BlockingRecv error: recv got EOF from 127.14.142.3:44035 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 21:24:12 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:12 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:12.269999 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:12.260161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:41359)
0926 21:24:12.260918 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:12.260949 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:12.261139 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:12.261526 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:12.261530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:12.262064 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:12.262272 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:12.262286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.263413 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.263420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:12.264405 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:12.264416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.265174 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.265180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:12.265364 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:12.266022 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:12.266040 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:12.269641 (+ 3601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":412,"threads_started":1}
W20260926 21:24:12.270125 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:12.260318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:41359 (local address 127.14.142.62:37799)
0926 21:24:12.260689 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:12.260700 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:12.261072 (+ 372us) server_negotiation.cc:407] Connection header received
0926 21:24:12.261284 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:12.261290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:12.261367 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:12.261476 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:12.262448 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.263264 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.264581 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.265007 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.265145 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:12.269869 (+ 4724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:41359: BlockingRecv error: recv got EOF from 127.14.142.2:41359 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 26 21:24:12 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:12 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:12.728601 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:12.717258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:38489)
0926 21:24:12.717820 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:12.717841 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:12.717955 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:12.718285 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:12.718289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:12.719169 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:12.719365 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:12.719375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.720751 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.720757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:12.722260 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:12.722274 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.723173 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.723184 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:12.723371 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:12.724301 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:12.724324 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:12.728230 (+ 3906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":334,"threads_started":1}
W20260926 21:24:12.728765 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:12.717322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:38489 (local address 127.14.142.62:37799)
0926 21:24:12.717524 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:12.717529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:12.717944 (+ 415us) server_negotiation.cc:407] Connection header received
0926 21:24:12.718051 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:12.718055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:12.718119 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:12.718209 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:12.719690 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.720451 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.722430 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:12.722882 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:12.723003 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:12.728473 (+ 5470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:38489: BlockingRecv error: recv got EOF from 127.14.142.1:38489 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 26 21:24:13 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:13 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:13.173688 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:13.152844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:56501)
0926 21:24:13.153556 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:13.153576 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:13.153745 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:13.154242 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:13.154247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:13.154990 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:13.155149 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:13.155159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.156861 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.156866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:13.157980 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:13.157994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.158938 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.158943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:13.159077 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:13.169221 (+ 10144us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:13.169246 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:13.173432 (+ 4186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":420,"threads_started":1}
W20260926 21:24:13.174142 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:13.153041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:56501 (local address 127.14.142.62:37799)
0926 21:24:13.153308 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:13.153315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:13.153784 (+ 469us) server_negotiation.cc:407] Connection header received
0926 21:24:13.153882 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:13.153888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:13.153984 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:13.154133 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:13.155454 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.156691 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.158302 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.158768 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.158885 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:13.173781 (+ 14896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:56501: BlockingRecv error: recv got EOF from 127.14.142.3:56501 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Sep 26 21:24:13 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:13 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
W20260926 21:24:13.282953 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:13.271503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43217 (local address 127.14.142.62:37799)
0926 21:24:13.271764 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:13.271769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:13.272309 (+ 540us) server_negotiation.cc:407] Connection header received
0926 21:24:13.272621 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:13.272625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:13.272704 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:13.272829 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:13.273904 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.274743 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.276248 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.276801 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.276955 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:13.282760 (+ 5805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:43217: BlockingRecv error: recv got EOF from 127.14.142.2:43217 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 21:24:13.282944 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:13.271220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:43217)
0926 21:24:13.272115 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:13.272136 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:13.272412 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:13.272871 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:13.272876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:13.273395 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:13.273582 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:13.273592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.274922 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.274929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:13.276067 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:13.276080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.276977 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.276984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:13.277177 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:13.277900 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:13.277919 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:13.282505 (+ 4586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":432,"threads_started":1}
Sep 26 21:24:13 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:13 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:13.740792 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:13.729717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:33163)
0926 21:24:13.730295 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:13.730311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:13.730430 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:13.730814 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:13.730819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:13.731264 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:13.731519 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:13.731533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.732872 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.732879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:13.733815 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:13.733826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.734564 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.734569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:13.734746 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:13.735799 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:13.735827 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:13.740387 (+ 4560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":351,"threads_started":1}
W20260926 21:24:13.740918 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:13.730029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:33163 (local address 127.14.142.62:37799)
0926 21:24:13.730219 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:13.730223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:13.730397 (+ 174us) server_negotiation.cc:407] Connection header received
0926 21:24:13.730586 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:13.730589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:13.730655 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:13.730748 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:13.731683 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.732744 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.733964 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:13.734401 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:13.734528 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:13.740661 (+ 6133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:33163: BlockingRecv error: recv got EOF from 127.14.142.1:33163 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 26 21:24:14 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:14 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:14.185237 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:14.174679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:42999)
0926 21:24:14.175199 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:14.175215 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:14.175325 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:14.175621 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:14.175624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:14.175987 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:14.176144 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:14.176152 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.177650 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.177654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:14.178629 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:14.178640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.179378 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.179383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:14.179557 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:14.180260 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:14.180279 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:14.184872 (+ 4593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":297,"threads_started":1}
W20260926 21:24:14.185546 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:14.174813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:42999 (local address 127.14.142.62:37799)
0926 21:24:14.175008 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:14.175013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:14.175297 (+ 284us) server_negotiation.cc:407] Connection header received
0926 21:24:14.175424 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:14.175428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:14.175494 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:14.175590 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:14.176312 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.177480 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.178781 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.179218 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.179356 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:14.185196 (+ 5840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:42999: BlockingRecv error: recv got EOF from 127.14.142.3:42999 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 21:24:14 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:14 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:14.294162 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:14.284130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:57051)
0926 21:24:14.284717 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:14.284739 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:14.284874 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:14.285201 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:14.285204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:14.285716 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:14.285949 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:14.285959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.287194 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.287202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:14.288183 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:14.288195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.288987 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.288993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:14.289178 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:14.289832 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:14.289849 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:14.293879 (+ 4030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":334,"threads_started":1}
W20260926 21:24:14.294423 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:14.284333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:57051 (local address 127.14.142.62:37799)
0926 21:24:14.284537 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:14.284542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:14.284859 (+ 317us) server_negotiation.cc:407] Connection header received
0926 21:24:14.284968 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:14.284972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:14.285040 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:14.285151 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:14.286130 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.287069 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.288328 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.288836 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.288971 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:14.294101 (+ 5130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:57051: BlockingRecv error: recv got EOF from 127.14.142.2:57051 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 21:24:14 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:14 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:14.754084 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:14.742512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:51607)
0926 21:24:14.743297 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:14.743316 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:14.743466 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:14.744030 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:14.744037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:14.744739 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:14.744982 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:14.744993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.746218 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.746226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:14.747175 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:14.747186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.747974 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.747980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:14.748176 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:14.749064 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:14.749084 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:14.753788 (+ 4704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":434,"threads_started":1}
W20260926 21:24:14.754278 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:14.742512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51607 (local address 127.14.142.62:37799)
0926 21:24:14.742837 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:14.742844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:14.743414 (+ 570us) server_negotiation.cc:407] Connection header received
0926 21:24:14.743587 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:14.743593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:14.743730 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:14.743858 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:14.745173 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.746052 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.747330 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:14.747759 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:14.747911 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:14.754025 (+ 6114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:51607: BlockingRecv error: recv got EOF from 127.14.142.1:51607 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Sep 26 21:24:15 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:15 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:15.197192 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:15.186351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:48817)
0926 21:24:15.186925 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:15.186944 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:15.187079 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:15.187582 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:15.187585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:15.188057 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:15.188268 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:15.188278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.189600 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.189604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:15.190797 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:15.190809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.191906 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.191919 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:15.192157 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:15.192843 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:15.192872 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:15.196950 (+ 4078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":339,"threads_started":1}
W20260926 21:24:15.197412 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:15.186516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48817 (local address 127.14.142.62:37799)
0926 21:24:15.186713 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:15.186718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:15.187119 (+ 401us) server_negotiation.cc:407] Connection header received
0926 21:24:15.187231 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:15.187236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:15.187365 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:15.187499 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:15.188532 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.189422 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.191012 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.191740 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.191847 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:15.197146 (+ 5299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:48817: BlockingRecv error: recv got EOF from 127.14.142.3:48817 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 21:24:15 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:15 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:15.305750 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:15.295213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:50951)
0926 21:24:15.295802 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:15.295818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:15.295933 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:15.296266 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:15.296269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:15.296752 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:15.296925 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:15.296933 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.298007 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.298010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:15.298916 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:15.298926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.299719 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.299723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:15.299871 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:15.300543 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:15.300580 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:15.305469 (+ 4889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":332,"threads_started":1}
W20260926 21:24:15.305924 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:15.295268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:50951 (local address 127.14.142.62:37799)
0926 21:24:15.295473 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:15.295479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:15.295892 (+ 413us) server_negotiation.cc:407] Connection header received
0926 21:24:15.296037 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:15.296040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:15.296138 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:15.296229 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:15.297080 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.297872 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.299041 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.299546 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.299652 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:15.305704 (+ 6052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:50951: BlockingRecv error: recv got EOF from 127.14.142.2:50951 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 26 21:24:15 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:15 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:15.766589 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:15.755289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:48657)
0926 21:24:15.755815 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:15.755831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:15.755948 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:15.756474 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:15.756479 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:15.757110 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:15.757321 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:15.757332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.758798 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.758805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:15.760131 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:15.760145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.760984 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.760990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:15.761143 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:15.761798 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:15.761817 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:15.766159 (+ 4342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":277,"threads_started":1}
W20260926 21:24:15.766781 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:15.755391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:48657 (local address 127.14.142.62:37799)
0926 21:24:15.755564 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:15.755568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:15.755909 (+ 341us) server_negotiation.cc:407] Connection header received
0926 21:24:15.756104 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:15.756108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:15.756185 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:15.756279 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:15.757528 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.758604 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.760282 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:15.760795 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:15.760914 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:15.766474 (+ 5560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:48657: BlockingRecv error: recv got EOF from 127.14.142.1:48657 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 21:24:16 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:16 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:16.210872 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:16.198489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:35557)
0926 21:24:16.199389 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:16.199410 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:16.199579 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:16.200141 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:16.200145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:16.200801 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:16.201016 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:16.201026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.202540 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.202546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:16.204256 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:16.204280 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.205287 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.205293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:16.205456 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:16.206240 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:16.206261 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:16.210605 (+ 4344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":559,"threads_started":1}
W20260926 21:24:16.211100 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:16.198722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:35557 (local address 127.14.142.62:37799)
0926 21:24:16.198931 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:16.198936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:16.199538 (+ 602us) server_negotiation.cc:407] Connection header received
0926 21:24:16.199716 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:16.199720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:16.199892 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:16.200059 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:16.201306 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.202359 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.204471 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.204999 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.205138 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:16.210846 (+ 5708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:35557: BlockingRecv error: recv got EOF from 127.14.142.3:35557 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Sep 26 21:24:16 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:16 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:16.315984 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:16.306824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:51953)
0926 21:24:16.307329 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:16.307355 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:16.307515 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:16.307883 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:16.307887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:16.308340 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:16.308484 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:16.308492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.309530 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.309538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:16.310449 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:16.310459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.311189 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.311195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:16.311367 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:16.312019 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:16.312036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:16.315620 (+ 3584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":253,"threads_started":1}
W20260926 21:24:16.316102 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:16.307089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:51953 (local address 127.14.142.62:37799)
0926 21:24:16.307286 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:16.307292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:16.307439 (+ 147us) server_negotiation.cc:407] Connection header received
0926 21:24:16.307641 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:16.307647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:16.307745 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:16.307852 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:16.308650 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.309380 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.310590 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.311044 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.311178 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:16.315841 (+ 4663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:51953: BlockingRecv error: recv got EOF from 127.14.142.2:51953 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 26 21:24:16 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:16 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:16.778023 24218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:16.768074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:43481)
0926 21:24:16.768778 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:16.768801 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:16.769310 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:16.769733 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:16.769739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:16.770140 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:16.770365 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:16.770375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.771362 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.771368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:16.772289 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:16.772300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.773063 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.773069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:16.773257 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:16.773887 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:16.773907 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:16.777601 (+ 3694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":365,"threads_started":1}
W20260926 21:24:16.778035 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:16.768128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:43481 (local address 127.14.142.62:37799)
0926 21:24:16.768360 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:16.768370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:16.769126 (+ 756us) server_negotiation.cc:407] Connection header received
0926 21:24:16.769524 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:16.769529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:16.769596 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:16.769698 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:16.770534 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.771219 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.772432 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:16.772914 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:16.773043 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:16.777822 (+ 4779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:43481: BlockingRecv error: recv got EOF from 127.14.142.1:43481 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 26 21:24:17 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:17 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.3@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:17.222985 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:17.212130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.3:57647)
0926 21:24:17.212705 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:17.212730 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:17.212860 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:17.213198 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:17.213201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:17.213672 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:17.213828 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:17.213837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.215298 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.215306 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:17.216744 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:17.216764 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.217942 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.217950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:17.218159 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.218824 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:17.218844 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:17.222734 (+ 3890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":305,"threads_started":1}
W20260926 21:24:17.223497 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:17.212204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:57647 (local address 127.14.142.62:37799)
0926 21:24:17.212495 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:17.212500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:17.212842 (+ 342us) server_negotiation.cc:407] Connection header received
0926 21:24:17.212957 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:17.212961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:17.213043 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:17.213138 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:17.214091 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.215026 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.216962 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.217726 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.217836 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.223017 (+ 5181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.3:57647: BlockingRecv error: recv got EOF from 127.14.142.3:57647 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Sep 26 21:24:17 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:17 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.2@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:17.327477 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:17.317498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.2:39225)
0926 21:24:17.318053 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:17.318069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:17.318239 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:17.318610 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:17.318614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:17.318992 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:17.319186 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:17.319194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.320236 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.320243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:17.321478 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:17.321494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.322302 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.322308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:17.322509 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.323213 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:17.323230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:17.327118 (+ 3888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":292,"threads_started":1}
W20260926 21:24:17.327762 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:17.317637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:39225 (local address 127.14.142.62:37799)
0926 21:24:17.317857 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:17.317865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:17.318168 (+ 303us) server_negotiation.cc:407] Connection header received
0926 21:24:17.318375 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:17.318379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:17.318461 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:17.318570 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:17.319333 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.320093 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.321680 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.322134 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.322274 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.327445 (+ 5171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.2:39225: BlockingRecv error: recv got EOF from 127.14.142.2:39225 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 21:24:17 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
Sep 26 21:24:17 dist-test-slave-4d5t krb5kdc[23172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.14.142.1@KRBTEST.COM for oryx/127.14.142.62@KRBTEST.COM, Server not found in Kerberos database
I20260926 21:24:17.791419 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:17.779364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37799 (local address 127.14.142.1:58171)
0926 21:24:17.779961 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:17.779977 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:17.780115 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:17.780728 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:17.780733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:17.781746 (+ 1013us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:17.782138 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:17.782153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.783360 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.783365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:17.784963 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:17.784979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.785804 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.785809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:17.785938 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.786663 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:17.786683 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:17.791107 (+ 4424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.14.142.62:37799: Server oryx/127.14.142.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":363,"threads_started":1}
W20260926 21:24:17.791838 23910 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:17.779441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:58171 (local address 127.14.142.62:37799)
0926 21:24:17.779720 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:17.779731 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:17.780076 (+ 345us) server_negotiation.cc:407] Connection header received
0926 21:24:17.780386 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:17.780391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:17.780505 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:17.780667 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:17.782305 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.783206 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.785180 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.785623 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.785737 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.791577 (+ 5840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.14.142.1:58171: BlockingRecv error: recv got EOF from 127.14.142.1:58171 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 21:24:17.896600 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23761
I20260926 21:24:17.910915 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:17.898007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.14.142.1:44803)
0926 21:24:17.898699 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:17.898708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:17.898739 (+ 31us) server_negotiation.cc:407] Connection header received
0926 21:24:17.898809 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:17.898815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:17.899072 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:17.899260 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:17.900979 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.902385 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.903788 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.904399 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.904607 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.907256 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:17.907274 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:17.907289 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:17.907328 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:17.910275 (+ 2947us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.142.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 21:24:17.910613 (+ 338us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54826: Request ticket server kudu/127.14.142.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":331,"threads_started":1}
W20260926 21:24:17.911036 24223 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54826: Request ticket server kudu/127.14.142.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 21:24:17.911168 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23761: Not authorized: Client connection negotiation failed: client connection to 127.14.142.1:44803: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.142.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 21:24:17.911381 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23761
I20260926 21:24:17.931170 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23912
I20260926 21:24:17.944784 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:17.932422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.14.142.2:43807)
0926 21:24:17.933195 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:17.933208 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:17.933242 (+ 34us) server_negotiation.cc:407] Connection header received
0926 21:24:17.933326 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:17.933338 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:17.933676 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:17.933902 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:17.935173 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.936284 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.938128 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.938686 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.938798 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.941449 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:17.941457 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:17.941472 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:17.941520 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:17.944086 (+ 2566us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.142.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 21:24:17.944433 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56502: Request ticket server kudu/127.14.142.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":401,"threads_started":1}
W20260926 21:24:17.944943 24224 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56502: Request ticket server kudu/127.14.142.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 21:24:17.944963 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23912: Not authorized: Client connection negotiation failed: client connection to 127.14.142.2:43807: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.142.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 21:24:17.945175 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23912
I20260926 21:24:17.964594 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24045
I20260926 21:24:17.979713 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:17.965916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58808 (local address 127.14.142.3:38395)
0926 21:24:17.966575 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:17.966582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:17.966609 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:24:17.966681 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:17.966689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:17.966938 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:17.967178 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:17.968622 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.969866 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.971119 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:17.971730 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:17.971878 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:17.975712 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:17.975722 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:17.975740 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:17.975802 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:17.978829 (+ 3027us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.142.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 21:24:17.979317 (+ 488us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58808: Request ticket server kudu/127.14.142.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":354,"threads_started":1}
W20260926 21:24:17.979859 24225 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58808: Request ticket server kudu/127.14.142.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 21:24:17.980067 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24045: Not authorized: Client connection negotiation failed: client connection to 127.14.142.3:38395: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.14.142.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 21:24:17.980273 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24045
I20260926 21:24:17.998518 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 23692
I20260926 21:24:18.000128 23754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:18.121088 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 23692
2026-09-26T21:24:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22233 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T21:24:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:24:18Z Disabled control of system clock
I20260926 21:24:18.205762 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44821
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 21:24:18.552124 24230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:18.552441 24230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:18.552521 24230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:18.559965 24230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:24:18.560110 24230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:18.560150 24230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:18.560201 24230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:24:18.560246 24230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:24:18.569860 24230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44821
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:18.571197 24230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:18.572968 24230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:18.585287 24235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:18.586153 24236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:18.586668 24238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:18.586838 24230 server_base.cc:1034] running on GCE node
I20260926 21:24:18.588245 24230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:18.590773 24230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:18.592115 24230 hybrid_clock.cc:648] HybridClock initialized: now 1790457858592040 us; error 53 us; skew 500 ppm
I20260926 21:24:18.592728 24230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:18.596753 24230 webserver.cc:460] Webserver started at http://127.14.142.62:37749/ using document root <none> and password file <none>
I20260926 21:24:18.597885 24230 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:18.598201 24230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:18.598599 24230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:18.604223 24230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "151ea092cb3140da9e3db35c1882e8c3"
format_stamp: "Formatted at 2026-09-26 21:24:18 on dist-test-slave-4d5t"
server_key: "6ab2da4c874a0d52d6065d056b9b5e44"
server_key_iv: "622258524cfff35f50ba53406f82debf"
server_key_version: "encryptionkey@0"
I20260926 21:24:18.605583 24230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "151ea092cb3140da9e3db35c1882e8c3"
format_stamp: "Formatted at 2026-09-26 21:24:18 on dist-test-slave-4d5t"
server_key: "6ab2da4c874a0d52d6065d056b9b5e44"
server_key_iv: "622258524cfff35f50ba53406f82debf"
server_key_version: "encryptionkey@0"
I20260926 21:24:18.614331 24230 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.004s
I20260926 21:24:18.619714 24244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:18.621831 24230 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 21:24:18.622169 24230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "151ea092cb3140da9e3db35c1882e8c3"
format_stamp: "Formatted at 2026-09-26 21:24:18 on dist-test-slave-4d5t"
server_key: "6ab2da4c874a0d52d6065d056b9b5e44"
server_key_iv: "622258524cfff35f50ba53406f82debf"
server_key_version: "encryptionkey@0"
I20260926 21:24:18.622515 24230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:18.643153 24230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:18.644093 24230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:18.644379 24230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:18.665998 24230 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:44821
I20260926 21:24:18.666021 24295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:44821 every 8 connection(s)
I20260926 21:24:18.667428 24230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:24:18.671428 24296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:24:18.675805 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24230
I20260926 21:24:18.676079 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:24:18.676914 14904 external_mini_cluster.cc:1428] Setting key 4098f066ad602778fc2c772f41b1746e
I20260926 21:24:18.683228 24296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3: Bootstrap starting.
I20260926 21:24:18.687048 24296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3: Neither blocks nor log segments found. Creating new log.
I20260926 21:24:18.688421 24296 log.cc:824] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:18.693691 24296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3: No bootstrap required, opened a new log
I20260926 21:24:18.695199 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:18.679336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.14.142.62:44821)
0926 21:24:18.681072 (+ 1736us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:18.681086 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:18.681142 (+ 56us) server_negotiation.cc:407] Connection header received
0926 21:24:18.683028 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:18.683052 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:18.683706 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:18.684090 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:18.684908 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:18.686475 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:18.688516 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:18.689886 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:18.690242 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:18.692307 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:18.692316 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:18.692336 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:18.693096 (+ 760us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:18.693196 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:18.693224 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:18.693344 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:18.693724 (+ 380us) server_negotiation.cc:299] Negotiation successful
0926 21:24:18.694103 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":617,"threads_started":1}
I20260926 21:24:18.702163 24296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151ea092cb3140da9e3db35c1882e8c3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44821 } }
I20260926 21:24:18.702574 24296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:24:18.702694 24296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151ea092cb3140da9e3db35c1882e8c3, State: Initialized, Role: FOLLOWER
I20260926 21:24:18.703337 24296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151ea092cb3140da9e3db35c1882e8c3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44821 } }
I20260926 21:24:18.703572 24296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:24:18.703740 24296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:24:18.703909 24296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:24:18.706564 24296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151ea092cb3140da9e3db35c1882e8c3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44821 } }
I20260926 21:24:18.707131 24296 leader_election.cc:302] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 151ea092cb3140da9e3db35c1882e8c3; no voters:
I20260926 21:24:18.707779 24296 leader_election.cc:288] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:24:18.707988 24301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:24:18.708802 24301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [term 1 LEADER]: Becoming Leader. State: Replica: 151ea092cb3140da9e3db35c1882e8c3, State: Running, Role: LEADER
I20260926 21:24:18.709393 24301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151ea092cb3140da9e3db35c1882e8c3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44821 } }
I20260926 21:24:18.710106 24296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:24:18.713137 24303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 151ea092cb3140da9e3db35c1882e8c3. Latest consensus state: current_term: 1 leader_uuid: "151ea092cb3140da9e3db35c1882e8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151ea092cb3140da9e3db35c1882e8c3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44821 } } }
I20260926 21:24:18.713902 24303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:18.714654 24302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "151ea092cb3140da9e3db35c1882e8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151ea092cb3140da9e3db35c1882e8c3" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44821 } } }
I20260926 21:24:18.715343 24302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3 [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:18.717788 24308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:24:18.722848 24308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:24:18.734681 24308 catalog_manager.cc:1347] Generated new cluster ID: ebcc52b3fa134354a7eaf00e0a4eaf01
I20260926 21:24:18.734834 24308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:24:18.759725 24308 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:24:18.761168 24308 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:24:18.776371 24308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 151ea092cb3140da9e3db35c1882e8c3: Generated new TSK 0
I20260926 21:24:18.777412 24308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:24:18.783196 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44821
--builtin_ntp_servers=127.14.142.20:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 21:24:19.153843 24320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:19.154166 24320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:19.154269 24320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:19.162339 24320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:19.162468 24320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:19.162577 24320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:24:19.172117 24320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:19.173473 24320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:19.175247 24320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:19.189320 24328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:19.189353 24325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:19.189569 24326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:19.190282 24320 server_base.cc:1034] running on GCE node
I20260926 21:24:19.190989 24320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:19.192163 24320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:19.193553 24320 hybrid_clock.cc:648] HybridClock initialized: now 1790457859193455 us; error 95 us; skew 500 ppm
I20260926 21:24:19.193974 24320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:19.197016 24320 webserver.cc:460] Webserver started at http://127.14.142.1:35811/ using document root <none> and password file <none>
I20260926 21:24:19.197854 24320 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:19.198006 24320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:19.198374 24320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:19.201673 24320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "538a8d4b3d194995861184a80f6e369b"
format_stamp: "Formatted at 2026-09-26 21:24:19 on dist-test-slave-4d5t"
server_key: "ba5f085792d3f5f7f8d3e412e86fba39"
server_key_iv: "e14c287ca593ad3ad325007394c18c7f"
server_key_version: "encryptionkey@0"
I20260926 21:24:19.202564 24320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "538a8d4b3d194995861184a80f6e369b"
format_stamp: "Formatted at 2026-09-26 21:24:19 on dist-test-slave-4d5t"
server_key: "ba5f085792d3f5f7f8d3e412e86fba39"
server_key_iv: "e14c287ca593ad3ad325007394c18c7f"
server_key_version: "encryptionkey@0"
I20260926 21:24:19.208808 24320 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 21:24:19.212988 24334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:19.214571 24320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:24:19.214795 24320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "538a8d4b3d194995861184a80f6e369b"
format_stamp: "Formatted at 2026-09-26 21:24:19 on dist-test-slave-4d5t"
server_key: "ba5f085792d3f5f7f8d3e412e86fba39"
server_key_iv: "e14c287ca593ad3ad325007394c18c7f"
server_key_version: "encryptionkey@0"
I20260926 21:24:19.215058 24320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:19.228459 24320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:19.229470 24320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:19.229852 24320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:19.231058 24320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:19.232820 24320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:19.232956 24320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:19.233114 24320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:19.233230 24320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:19.273391 24320 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:36091
I20260926 21:24:19.273439 24446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:36091 every 8 connection(s)
I20260926 21:24:19.274791 24320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:24:19.282183 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24320
I20260926 21:24:19.282594 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:24:19.283429 14904 external_mini_cluster.cc:1428] Setting key 9075227db8f9dfddd2f9ce38c2459013
I20260926 21:24:19.286651 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44821
--builtin_ntp_servers=127.14.142.20:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 21:24:19.289741 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:19.277737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:56655 (local address 127.14.142.62:44821)
0926 21:24:19.278241 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:19.278246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:19.279278 (+ 1032us) server_negotiation.cc:407] Connection header received
0926 21:24:19.281343 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:19.281349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:19.281467 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:19.281601 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:19.282927 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.283818 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.285935 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.286416 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.286564 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:19.288602 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:19.288614 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:19.288622 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:19.288656 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:19.288722 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:19.288738 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:19.288853 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:19.289128 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 21:24:19.289365 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":285,"threads_started":1}
I20260926 21:24:19.290910 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:19.278135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44821 (local address 127.14.142.1:56655)
0926 21:24:19.279116 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:19.279153 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:19.281022 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:19.281867 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:19.281877 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:19.281986 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:19.282722 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:19.282752 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.284004 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.284015 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:19.285758 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:19.285772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.286683 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.286692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:19.287026 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:19.287981 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:24:19.287985 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:19.288051 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:24:19.288193 (+ 142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:19.288855 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:19.288870 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:19.288882 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 21:24:19.289121 (+ 239us) client_negotiation.cc:240] Negotiation successful
0926 21:24:19.289730 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":299,"threads_started":1}
I20260926 21:24:19.292539 24447 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44821
I20260926 21:24:19.292905 24447 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:19.293704 24447 heartbeater.cc:511] Master 127.14.142.62:44821 requested a full tablet report, sending...
I20260926 21:24:19.295861 24261 ts_manager.cc:194] Registered new tserver with Master: 538a8d4b3d194995861184a80f6e369b (127.14.142.1:36091)
I20260926 21:24:19.297428 24261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.1:56655
W20260926 21:24:19.647364 24451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:19.647750 24451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:19.647935 24451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:19.656445 24451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:19.656646 24451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:19.656829 24451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:24:19.669746 24451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:19.671378 24451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:19.673741 24451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:19.688830 24456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:19.688992 24457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:19.689469 24459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:19.690104 24451 server_base.cc:1034] running on GCE node
I20260926 21:24:19.690732 24451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:19.691761 24451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:19.692976 24451 hybrid_clock.cc:648] HybridClock initialized: now 1790457859692924 us; error 42 us; skew 500 ppm
I20260926 21:24:19.693589 24451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:19.697172 24451 webserver.cc:460] Webserver started at http://127.14.142.2:44349/ using document root <none> and password file <none>
I20260926 21:24:19.698212 24451 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:19.698382 24451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:19.698841 24451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:19.702870 24451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "26c5a203e97943d88de5dec137576a03"
format_stamp: "Formatted at 2026-09-26 21:24:19 on dist-test-slave-4d5t"
server_key: "2be328dff3a5d36a4a00178552ff0afc"
server_key_iv: "e3b10dccc50138eade6e3fad9140d3f1"
server_key_version: "encryptionkey@0"
I20260926 21:24:19.703884 24451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "26c5a203e97943d88de5dec137576a03"
format_stamp: "Formatted at 2026-09-26 21:24:19 on dist-test-slave-4d5t"
server_key: "2be328dff3a5d36a4a00178552ff0afc"
server_key_iv: "e3b10dccc50138eade6e3fad9140d3f1"
server_key_version: "encryptionkey@0"
I20260926 21:24:19.709287 24451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 21:24:19.713129 24465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:19.714706 24451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:24:19.714895 24451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "26c5a203e97943d88de5dec137576a03"
format_stamp: "Formatted at 2026-09-26 21:24:19 on dist-test-slave-4d5t"
server_key: "2be328dff3a5d36a4a00178552ff0afc"
server_key_iv: "e3b10dccc50138eade6e3fad9140d3f1"
server_key_version: "encryptionkey@0"
I20260926 21:24:19.715133 24451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:19.733752 24451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:19.734714 24451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:19.735031 24451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:19.736007 24451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:19.737547 24451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:19.737672 24451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:19.737815 24451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:19.737908 24451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:19.775097 24451 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:40291
I20260926 21:24:19.775282 24577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:40291 every 8 connection(s)
I20260926 21:24:19.776470 24451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:24:19.785686 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24451
I20260926 21:24:19.785902 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:24:19.786437 14904 external_mini_cluster.cc:1428] Setting key 01c902f5d98ff940602a3daf78d520d6
I20260926 21:24:19.789808 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44821
--builtin_ntp_servers=127.14.142.20:37785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 21:24:19.792276 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:19.779419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:59459 (local address 127.14.142.62:44821)
0926 21:24:19.779674 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:19.779681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:19.781498 (+ 1817us) server_negotiation.cc:407] Connection header received
0926 21:24:19.783928 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:19.783932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:19.784011 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:19.784109 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:19.785628 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.786761 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.788858 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.789468 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.789626 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:19.791181 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:19.791190 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:19.791197 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:19.791225 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:19.791271 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:19.791291 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:19.791423 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:19.791751 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 21:24:19.791871 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 21:24:19.793359 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:19.779937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44821 (local address 127.14.142.2:59459)
0926 21:24:19.781273 (+ 1336us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:19.781324 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:19.783580 (+ 2256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:19.784384 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:19.784393 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:19.784498 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:19.785357 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:19.785403 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.786990 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.786995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:19.788667 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:19.788696 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:19.789732 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:19.789738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:19.790001 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:19.790807 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:24:19.790810 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:19.790855 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:24:19.790944 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:19.791455 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:19.791468 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:19.791477 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 21:24:19.791707 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 21:24:19.792082 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":390,"threads_started":1}
I20260926 21:24:19.794986 24578 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44821
I20260926 21:24:19.795320 24578 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:19.796118 24578 heartbeater.cc:511] Master 127.14.142.62:44821 requested a full tablet report, sending...
I20260926 21:24:19.797924 24261 ts_manager.cc:194] Registered new tserver with Master: 26c5a203e97943d88de5dec137576a03 (127.14.142.2:40291)
I20260926 21:24:19.798776 24261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.2:59459
W20260926 21:24:20.155429 24581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:20.155818 24581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:20.155987 24581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:20.163444 24581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:20.163625 24581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:20.163822 24581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:24:20.173834 24581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:37785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44821
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:20.175220 24581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:20.177047 24581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:20.191355 24587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:20.191469 24586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:20.191386 24589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:20.192646 24581 server_base.cc:1034] running on GCE node
I20260926 21:24:20.193337 24581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:20.194464 24581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:20.195699 24581 hybrid_clock.cc:648] HybridClock initialized: now 1790457860195623 us; error 65 us; skew 500 ppm
I20260926 21:24:20.196076 24581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:20.199507 24581 webserver.cc:460] Webserver started at http://127.14.142.3:46537/ using document root <none> and password file <none>
I20260926 21:24:20.200488 24581 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:20.200711 24581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:20.201100 24581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:20.204321 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "bfe0e4d7f834489f92267c69625ede11"
format_stamp: "Formatted at 2026-09-26 21:24:20 on dist-test-slave-4d5t"
server_key: "14b89e11aeb7c7334cf48284a0a0572c"
server_key_iv: "e951e471ef4309cedc886d296594e41e"
server_key_version: "encryptionkey@0"
I20260926 21:24:20.205346 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "bfe0e4d7f834489f92267c69625ede11"
format_stamp: "Formatted at 2026-09-26 21:24:20 on dist-test-slave-4d5t"
server_key: "14b89e11aeb7c7334cf48284a0a0572c"
server_key_iv: "e951e471ef4309cedc886d296594e41e"
server_key_version: "encryptionkey@0"
I20260926 21:24:20.212728 24581 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260926 21:24:20.216815 24595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:20.218423 24581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:24:20.218734 24581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "bfe0e4d7f834489f92267c69625ede11"
format_stamp: "Formatted at 2026-09-26 21:24:20 on dist-test-slave-4d5t"
server_key: "14b89e11aeb7c7334cf48284a0a0572c"
server_key_iv: "e951e471ef4309cedc886d296594e41e"
server_key_version: "encryptionkey@0"
I20260926 21:24:20.219019 24581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:20.255198 24581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:20.256091 24581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:20.256413 24581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:20.257491 24581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:20.259012 24581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:20.259140 24581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:20.259276 24581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:20.259369 24581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:20.298866 24581 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:33313
I20260926 21:24:20.298928 24707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:33313 every 8 connection(s)
I20260926 21:24:20.300375 24581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:24:20.300500 24447 heartbeater.cc:503] Master 127.14.142.62:44821 was elected leader, sending a full tablet report...
I20260926 21:24:20.308938 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24581
I20260926 21:24:20.309150 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:24:20.309679 14904 external_mini_cluster.cc:1428] Setting key 3e92b43b849ded1966dea8ae8a8a7d06
I20260926 21:24:20.316917 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:20.303780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:53419 (local address 127.14.142.62:44821)
0926 21:24:20.304339 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:20.304347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:20.306673 (+ 2326us) server_negotiation.cc:407] Connection header received
0926 21:24:20.308533 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:20.308539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:20.308812 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:20.308934 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:20.310518 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.311598 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.313695 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.314108 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.314215 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:20.315628 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:20.315633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:20.315636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:20.315654 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:20.315702 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:20.315721 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:20.315797 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:20.316144 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 21:24:20.316385 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":258,"threads_started":1}
I20260926 21:24:20.318010 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:20.304908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44821 (local address 127.14.142.3:53419)
0926 21:24:20.306439 (+ 1531us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:20.306500 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:20.308204 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:20.309267 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:20.309280 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:20.309408 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:20.310309 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:20.310337 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.311777 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.311781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:20.313529 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:20.313541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.314317 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.314322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:20.314523 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:20.315264 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:24:20.315267 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:20.315311 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:24:20.315400 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:20.315819 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:20.315827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:20.315833 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 21:24:20.316010 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 21:24:20.316522 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":468,"threads_started":1}
I20260926 21:24:20.319769 24708 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44821
I20260926 21:24:20.320119 24708 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:20.320959 24708 heartbeater.cc:511] Master 127.14.142.62:44821 requested a full tablet report, sending...
I20260926 21:24:20.322503 24261 ts_manager.cc:194] Registered new tserver with Master: bfe0e4d7f834489f92267c69625ede11 (127.14.142.3:33313)
I20260926 21:24:20.323448 24261 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.3:53419
I20260926 21:24:20.326122 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 21:24:20.334772 24710 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:20.331191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.14.142.62:44821)
0926 21:24:20.331465 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:20.331471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:20.333317 (+ 1846us) server_negotiation.cc:407] Connection header received
0926 21:24:20.333444 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:20.333448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:20.333538 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:20.333651 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:20.334409 (+ 758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34834: BlockingRecv error: recv got EOF from 127.0.0.1:34834 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260926 21:24:20.335798 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24320
I20260926 21:24:20.344937 24718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:20.336978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.14.142.1:36091)
0926 21:24:20.337657 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:20.337664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:20.337692 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:24:20.337757 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:20.337763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:20.337994 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:20.338142 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:20.338735 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.339984 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.341477 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.342124 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.342231 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:20.343471 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:20.343479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:20.343498 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:20.344115 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:20.344168 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:20.344188 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:20.344290 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:20.344437 (+ 147us) server_negotiation.cc:299] Negotiation successful
0926 21:24:20.344628 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":362,"threads_started":1}
I20260926 21:24:20.346306 24442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:20.431523 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24320
I20260926 21:24:20.450237 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24451
I20260926 21:24:20.458355 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:20.451302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.14.142.2:40291)
0926 21:24:20.451815 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:20.451822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:20.451848 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:24:20.451914 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:20.451921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:20.452131 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:20.452306 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:20.452731 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.453707 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.454956 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.455539 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.455694 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:20.456870 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:20.456878 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:20.456893 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:20.457502 (+ 609us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:20.457550 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:20.457575 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:20.457715 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:20.457851 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 21:24:20.458004 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":242,"threads_started":1}
I20260926 21:24:20.459743 24573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:20.545802 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24451
I20260926 21:24:20.562258 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24581
I20260926 21:24:20.571764 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:20.563504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.14.142.3:33313)
0926 21:24:20.564174 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:20.564184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:20.564217 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:24:20.564279 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:20.564285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:20.564528 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:20.564782 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:20.565314 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.566495 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.567956 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:20.568801 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:20.568923 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:20.570000 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:20.570010 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:20.570033 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:20.570713 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:20.570767 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:20.570790 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:20.570902 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:20.571281 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 21:24:20.571409 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":326,"threads_started":1}
I20260926 21:24:20.573285 24703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:20.658996 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24581
I20260926 21:24:20.676036 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24230
I20260926 21:24:20.677351 24291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:20.783020 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24230
2026-09-26T21:24:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2656 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T21:24:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:24:20Z Disabled control of system clock
I20260926 21:24:20.871507 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:37969
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:37969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 21:24:21.233309 24728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:21.233636 24728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:21.233794 24728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:21.241761 24728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:24:21.241906 24728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:21.242004 24728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:21.242122 24728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:24:21.242208 24728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:24:21.254025 24728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:37969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:37969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:21.255702 24728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:21.257782 24728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:21.271644 24733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:21.271782 24736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:21.271728 24734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:21.272928 24728 server_base.cc:1034] running on GCE node
I20260926 21:24:21.273805 24728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:21.275336 24728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:21.276643 24728 hybrid_clock.cc:648] HybridClock initialized: now 1790457861276540 us; error 42 us; skew 500 ppm
I20260926 21:24:21.277048 24728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:21.279994 24728 webserver.cc:460] Webserver started at http://127.14.142.62:40139/ using document root <none> and password file <none>
I20260926 21:24:21.280841 24728 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:21.280938 24728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:21.281324 24728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:21.284389 24728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "0c88972a8a7e43998586d54bc41c4d4d"
format_stamp: "Formatted at 2026-09-26 21:24:21 on dist-test-slave-4d5t"
server_key: "94ed131e677a175824085a2c9e7aa818"
server_key_iv: "89d900c918b4673fd49192dad9c4150c"
server_key_version: "encryptionkey@0"
I20260926 21:24:21.285312 24728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "0c88972a8a7e43998586d54bc41c4d4d"
format_stamp: "Formatted at 2026-09-26 21:24:21 on dist-test-slave-4d5t"
server_key: "94ed131e677a175824085a2c9e7aa818"
server_key_iv: "89d900c918b4673fd49192dad9c4150c"
server_key_version: "encryptionkey@0"
I20260926 21:24:21.290647 24728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:24:21.294670 24742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:21.296204 24728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:24:21.296414 24728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "0c88972a8a7e43998586d54bc41c4d4d"
format_stamp: "Formatted at 2026-09-26 21:24:21 on dist-test-slave-4d5t"
server_key: "94ed131e677a175824085a2c9e7aa818"
server_key_iv: "89d900c918b4673fd49192dad9c4150c"
server_key_version: "encryptionkey@0"
I20260926 21:24:21.296685 24728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:21.302856 24728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:21.303761 24728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:21.304095 24728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:21.326051 24728 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:37969
I20260926 21:24:21.326069 24793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:37969 every 8 connection(s)
I20260926 21:24:21.327490 24728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:24:21.329784 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24728
I20260926 21:24:21.330010 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:24:21.330734 14904 external_mini_cluster.cc:1428] Setting key bec739344d503d720e227006b4508232
I20260926 21:24:21.331624 24794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:24:21.340646 24797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:21.333449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.14.142.62:37969)
0926 21:24:21.334750 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:21.334763 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:21.334807 (+ 44us) server_negotiation.cc:407] Connection header received
0926 21:24:21.336667 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:21.336692 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:21.337189 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:21.337578 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:21.338770 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:21.338776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:21.338790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:21.338806 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:21.338883 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:21.338907 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:21.339132 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:21.339240 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 21:24:21.339570 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":554,"threads_started":1}
I20260926 21:24:21.346122 24794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d: Bootstrap starting.
I20260926 21:24:21.349746 24794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d: Neither blocks nor log segments found. Creating new log.
I20260926 21:24:21.350818 24794 log.cc:824] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:21.354043 24794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d: No bootstrap required, opened a new log
I20260926 21:24:21.361878 24794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c88972a8a7e43998586d54bc41c4d4d" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37969 } }
I20260926 21:24:21.362257 24794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:24:21.362321 24794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c88972a8a7e43998586d54bc41c4d4d, State: Initialized, Role: FOLLOWER
I20260926 21:24:21.362990 24794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c88972a8a7e43998586d54bc41c4d4d" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37969 } }
I20260926 21:24:21.363281 24794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:24:21.363430 24794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:24:21.363615 24794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:24:21.366950 24794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c88972a8a7e43998586d54bc41c4d4d" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37969 } }
I20260926 21:24:21.368160 24794 leader_election.cc:302] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c88972a8a7e43998586d54bc41c4d4d; no voters:
I20260926 21:24:21.368903 24794 leader_election.cc:288] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:24:21.369166 24799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:24:21.370375 24799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [term 1 LEADER]: Becoming Leader. State: Replica: 0c88972a8a7e43998586d54bc41c4d4d, State: Running, Role: LEADER
I20260926 21:24:21.371134 24799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c88972a8a7e43998586d54bc41c4d4d" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37969 } }
I20260926 21:24:21.371546 24794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:24:21.374127 24801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c88972a8a7e43998586d54bc41c4d4d. Latest consensus state: current_term: 1 leader_uuid: "0c88972a8a7e43998586d54bc41c4d4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c88972a8a7e43998586d54bc41c4d4d" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37969 } } }
I20260926 21:24:21.374585 24801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:21.375204 24800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c88972a8a7e43998586d54bc41c4d4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c88972a8a7e43998586d54bc41c4d4d" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 37969 } } }
I20260926 21:24:21.375554 24800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:21.378140 24806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:24:21.384327 24806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:24:21.392620 24806 catalog_manager.cc:1347] Generated new cluster ID: e2cfb2906b934c469bc6c297d96720a9
I20260926 21:24:21.392757 24806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:24:21.409265 24806 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:24:21.409782 24806 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:24:21.415318 24806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c88972a8a7e43998586d54bc41c4d4d: Generated new TSK 0
I20260926 21:24:21.416234 24806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:24:21.428943 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37969
--builtin_ntp_servers=127.14.142.20:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 21:24:21.781412 24819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:21.781700 24819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:21.781793 24819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:21.789613 24819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:21.789736 24819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:21.789851 24819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:24:21.801239 24819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:21.802698 24819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:21.804504 24819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:21.818694 24824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:21.818770 24825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:21.818852 24827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:21.820058 24819 server_base.cc:1034] running on GCE node
I20260926 21:24:21.820828 24819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:21.821838 24819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:21.823068 24819 hybrid_clock.cc:648] HybridClock initialized: now 1790457861823025 us; error 125 us; skew 500 ppm
I20260926 21:24:21.823446 24819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:21.826151 24819 webserver.cc:460] Webserver started at http://127.14.142.1:41427/ using document root <none> and password file <none>
I20260926 21:24:21.826920 24819 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:21.827014 24819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:21.827344 24819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:21.830605 24819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "ebe0a5a002e649719bcef7028891d988"
format_stamp: "Formatted at 2026-09-26 21:24:21 on dist-test-slave-4d5t"
server_key: "297ee7ae15ec9d2e35d39ea6c5ec25ea"
server_key_iv: "7b11e572aec77f00378cf03bbb0014d0"
server_key_version: "encryptionkey@0"
I20260926 21:24:21.831486 24819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "ebe0a5a002e649719bcef7028891d988"
format_stamp: "Formatted at 2026-09-26 21:24:21 on dist-test-slave-4d5t"
server_key: "297ee7ae15ec9d2e35d39ea6c5ec25ea"
server_key_iv: "7b11e572aec77f00378cf03bbb0014d0"
server_key_version: "encryptionkey@0"
I20260926 21:24:21.838063 24819 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 21:24:21.843048 24833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:21.845202 24819 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 21:24:21.845427 24819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "ebe0a5a002e649719bcef7028891d988"
format_stamp: "Formatted at 2026-09-26 21:24:21 on dist-test-slave-4d5t"
server_key: "297ee7ae15ec9d2e35d39ea6c5ec25ea"
server_key_iv: "7b11e572aec77f00378cf03bbb0014d0"
server_key_version: "encryptionkey@0"
I20260926 21:24:21.845707 24819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:21.852785 24819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:21.853785 24819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:21.854077 24819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:21.855082 24819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:21.856667 24819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:21.856791 24819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:21.856930 24819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:21.857017 24819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:21.895097 24819 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:39317
I20260926 21:24:21.895215 24945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:39317 every 8 connection(s)
I20260926 21:24:21.896626 24819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:24:21.905166 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24819
I20260926 21:24:21.905408 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:24:21.906102 14904 external_mini_cluster.cc:1428] Setting key 0354cd843fc6b7041ff9b48cefc60fc0
I20260926 21:24:21.906236 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:21.899640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:55031 (local address 127.14.142.62:37969)
0926 21:24:21.900389 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:21.900397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:21.901768 (+ 1371us) server_negotiation.cc:407] Connection header received
0926 21:24:21.903592 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:21.903597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:21.903660 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:21.903786 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:21.905345 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:21.905351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:21.905356 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:21.905384 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:21.905435 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:21.905447 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:21.905595 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:21.905835 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 21:24:21.905910 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":373,"threads_started":1}
I20260926 21:24:21.907537 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:21.900264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37969 (local address 127.14.142.1:55031)
0926 21:24:21.901558 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:21.901606 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:21.903286 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:21.904017 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:21.904025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:21.904140 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:21.905004 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:24:21.905008 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:21.905061 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:24:21.905154 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:21.905597 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:21.905610 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:21.905621 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 21:24:21.905927 (+ 306us) client_negotiation.cc:240] Negotiation successful
0926 21:24:21.906201 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":459,"threads_started":1}
I20260926 21:24:21.909209 24946 heartbeater.cc:347] Connected to a master server at 127.14.142.62:37969
I20260926 21:24:21.909472 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37969
--builtin_ntp_servers=127.14.142.20:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 21:24:21.909562 24946 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:21.910244 24946 heartbeater.cc:511] Master 127.14.142.62:37969 requested a full tablet report, sending...
I20260926 21:24:21.913159 24759 ts_manager.cc:194] Registered new tserver with Master: ebe0a5a002e649719bcef7028891d988 (127.14.142.1:39317)
W20260926 21:24:22.271986 24950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:22.272495 24950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:22.272710 24950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:22.280471 24950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:22.280675 24950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:22.280874 24950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:24:22.290606 24950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:22.291956 24950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:22.293875 24950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:22.309131 24956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:22.309335 24955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:22.309361 24958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:22.314528 24950 server_base.cc:1034] running on GCE node
I20260926 21:24:22.315263 24950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:22.316380 24950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:22.317633 24950 hybrid_clock.cc:648] HybridClock initialized: now 1790457862317575 us; error 47 us; skew 500 ppm
I20260926 21:24:22.318005 24950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:22.320709 24950 webserver.cc:460] Webserver started at http://127.14.142.2:35927/ using document root <none> and password file <none>
I20260926 21:24:22.321466 24950 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:22.321586 24950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:22.321938 24950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:22.325186 24950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "96a213cbc21a4fa9846213ae3245012b"
format_stamp: "Formatted at 2026-09-26 21:24:22 on dist-test-slave-4d5t"
server_key: "11d4ef1a15936b815289b19e3b2db6ae"
server_key_iv: "183ecfd88d91204eddb165e41b734920"
server_key_version: "encryptionkey@0"
I20260926 21:24:22.326251 24950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "96a213cbc21a4fa9846213ae3245012b"
format_stamp: "Formatted at 2026-09-26 21:24:22 on dist-test-slave-4d5t"
server_key: "11d4ef1a15936b815289b19e3b2db6ae"
server_key_iv: "183ecfd88d91204eddb165e41b734920"
server_key_version: "encryptionkey@0"
I20260926 21:24:22.333163 24950 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:24:22.337625 24964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:22.339329 24950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:24:22.339505 24950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "96a213cbc21a4fa9846213ae3245012b"
format_stamp: "Formatted at 2026-09-26 21:24:22 on dist-test-slave-4d5t"
server_key: "11d4ef1a15936b815289b19e3b2db6ae"
server_key_iv: "183ecfd88d91204eddb165e41b734920"
server_key_version: "encryptionkey@0"
I20260926 21:24:22.339766 24950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:22.347285 24950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:22.348181 24950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:22.348402 24950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:22.349377 24950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:22.350991 24950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:22.351100 24950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:22.351234 24950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:22.351325 24950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:22.388761 24950 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:35507
I20260926 21:24:22.388824 25076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:35507 every 8 connection(s)
I20260926 21:24:22.390172 24950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:24:22.398329 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 24950
I20260926 21:24:22.398530 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:24:22.399008 14904 external_mini_cluster.cc:1428] Setting key 3bfec5303fb941ab78a39bb411079c84
I20260926 21:24:22.399587 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:22.393544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43239 (local address 127.14.142.62:37969)
0926 21:24:22.393760 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:22.393765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:22.395249 (+ 1484us) server_negotiation.cc:407] Connection header received
0926 21:24:22.397062 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:22.397068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:22.397167 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:22.397292 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:22.398832 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:22.398844 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:22.398848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:22.398865 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:22.398895 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:22.398906 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:22.399037 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:22.399231 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 21:24:22.399305 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 21:24:22.400841 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:22.394131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37969 (local address 127.14.142.2:43239)
0926 21:24:22.395071 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:22.395107 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:22.396783 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:22.397532 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:22.397540 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:22.397634 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:22.398456 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:24:22.398461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:22.398538 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:24:22.398655 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:22.399003 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:22.399013 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:22.399021 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 21:24:22.399321 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 21:24:22.399588 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":303,"threads_started":1}
I20260926 21:24:22.402285 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37969
--builtin_ntp_servers=127.14.142.20:46637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 21:24:22.402339 25077 heartbeater.cc:347] Connected to a master server at 127.14.142.62:37969
I20260926 21:24:22.402664 25077 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:22.403278 25077 heartbeater.cc:511] Master 127.14.142.62:37969 requested a full tablet report, sending...
I20260926 21:24:22.404853 24759 ts_manager.cc:194] Registered new tserver with Master: 96a213cbc21a4fa9846213ae3245012b (127.14.142.2:35507)
W20260926 21:24:22.746488 25080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:22.746795 25080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:22.746888 25080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:22.754590 25080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:22.754715 25080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:22.754835 25080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:24:22.764146 25080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:37969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:22.765509 25080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:22.767243 25080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:22.781505 25086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:22.781610 25088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:22.781576 25085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:22.782459 25080 server_base.cc:1034] running on GCE node
I20260926 21:24:22.783071 25080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:22.784107 25080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:22.785346 25080 hybrid_clock.cc:648] HybridClock initialized: now 1790457862785289 us; error 42 us; skew 500 ppm
I20260926 21:24:22.785766 25080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:24:22.788740 25080 webserver.cc:460] Webserver started at http://127.14.142.3:45795/ using document root <none> and password file <none>
I20260926 21:24:22.789652 25080 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:22.789773 25080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:22.790165 25080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:22.794549 25080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "ac0153d2b4ed4c12bc711c23bfd0c38b"
format_stamp: "Formatted at 2026-09-26 21:24:22 on dist-test-slave-4d5t"
server_key: "e17fa14f08f6e6f737687485f75628ab"
server_key_iv: "cedeadc33f3d4c97f96b7b0b59a6419f"
server_key_version: "encryptionkey@0"
I20260926 21:24:22.795651 25080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "ac0153d2b4ed4c12bc711c23bfd0c38b"
format_stamp: "Formatted at 2026-09-26 21:24:22 on dist-test-slave-4d5t"
server_key: "e17fa14f08f6e6f737687485f75628ab"
server_key_iv: "cedeadc33f3d4c97f96b7b0b59a6419f"
server_key_version: "encryptionkey@0"
I20260926 21:24:22.802798 25080 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260926 21:24:22.807045 25094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:22.808916 25080 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 21:24:22.809115 25080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "ac0153d2b4ed4c12bc711c23bfd0c38b"
format_stamp: "Formatted at 2026-09-26 21:24:22 on dist-test-slave-4d5t"
server_key: "e17fa14f08f6e6f737687485f75628ab"
server_key_iv: "cedeadc33f3d4c97f96b7b0b59a6419f"
server_key_version: "encryptionkey@0"
I20260926 21:24:22.809386 25080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:22.816720 25080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:22.817701 25080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:22.817967 25080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:22.819139 25080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:22.820935 25080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:22.821030 25080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:22.821111 25080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:22.821167 25080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:22.861414 25080 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:37529
I20260926 21:24:22.861474 25206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:37529 every 8 connection(s)
I20260926 21:24:22.862804 25080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:24:22.869730 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 25080
I20260926 21:24:22.869937 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:24:22.870384 14904 external_mini_cluster.cc:1428] Setting key cb558b6522dcccdd1d425eafdd7c0281
I20260926 21:24:22.872725 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:22.866234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:53737 (local address 127.14.142.62:37969)
0926 21:24:22.866449 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:22.866454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:22.867784 (+ 1330us) server_negotiation.cc:407] Connection header received
0926 21:24:22.869967 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:22.869974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:22.870073 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:22.870197 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:22.871765 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:22.871771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:22.871798 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:22.871821 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:22.871859 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:22.871882 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:22.872018 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:22.872263 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 21:24:22.872355 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 21:24:22.873860 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:22.866579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:37969 (local address 127.14.142.3:53737)
0926 21:24:22.867601 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:22.867646 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:22.869671 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:22.870438 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:22.870447 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:22.870595 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:22.871385 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:24:22.871388 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:22.871428 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:24:22.871519 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:22.871989 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:22.872000 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:22.872009 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 21:24:22.872360 (+ 351us) client_negotiation.cc:240] Negotiation successful
0926 21:24:22.872659 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":303,"threads_started":1}
I20260926 21:24:22.875607 25207 heartbeater.cc:347] Connected to a master server at 127.14.142.62:37969
I20260926 21:24:22.876173 25207 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:22.876873 25207 heartbeater.cc:511] Master 127.14.142.62:37969 requested a full tablet report, sending...
I20260926 21:24:22.878311 24759 ts_manager.cc:194] Registered new tserver with Master: ac0153d2b4ed4c12bc711c23bfd0c38b (127.14.142.3:37529)
I20260926 21:24:22.885941 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 21:24:22.892761 24948 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:24:22.890924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.14.142.62:37969)
0926 21:24:22.891093 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:22.891098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:22.891577 (+ 479us) server_negotiation.cc:407] Connection header received
0926 21:24:22.891705 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:22.891709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:22.891763 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:22.891858 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:22.892439 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36522: BlockingRecv error: recv got EOF from 127.0.0.1:36522 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 21:24:22.893239 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24819
I20260926 21:24:22.897665 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:22.894555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.14.142.1:39317)
0926 21:24:22.895168 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:22.895179 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:22.895207 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:24:22.895374 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:22.895381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:22.895587 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:22.895719 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:22.896847 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:22.896855 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:22.896866 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:22.896882 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:22.896933 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:22.896948 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:22.897090 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:22.897206 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 21:24:22.897282 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":338,"threads_started":1}
I20260926 21:24:22.898831 24941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:22.984349 24946 heartbeater.cc:503] Master 127.14.142.62:37969 was elected leader, sending a full tablet report...
I20260926 21:24:22.985659 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24819
W20260926 21:24:23.003300 24744 connection.cc:535] server connection from 127.14.142.1:55031 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 21:24:23.003479 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24950
I20260926 21:24:23.007542 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:23.004676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.14.142.2:35507)
0926 21:24:23.005225 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:23.005232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:23.005259 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:24:23.005334 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:23.005340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:23.005564 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:23.005706 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:23.006807 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:23.006813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:23.006825 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:23.006842 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:23.006881 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:23.006895 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:23.007008 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:23.007092 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 21:24:23.007225 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":272,"threads_started":1}
I20260926 21:24:23.008890 25072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:23.097203 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24950
I20260926 21:24:23.114476 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 25080
I20260926 21:24:23.119091 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:23.115742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.14.142.3:37529)
0926 21:24:23.116647 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:23.116657 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:23.116696 (+ 39us) server_negotiation.cc:407] Connection header received
0926 21:24:23.116761 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:23.116781 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:23.117006 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:23.117186 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:23.118314 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:23.118320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:23.118331 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:24:23.118347 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:23.118384 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:24:23.118396 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:23.118580 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:23.118654 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 21:24:23.118753 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":509,"threads_started":1}
I20260926 21:24:23.120395 25202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:23.204728 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 25080
I20260926 21:24:23.224051 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 24728
I20260926 21:24:23.225293 24789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:23.332008 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 24728
2026-09-26T21:24:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2546 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:24:23 dist-test-slave-4d5t krb5kdc[25225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:24:23 dist-test-slave-4d5t krb5kdc[25225](info): set up 2 sockets
Sep 26 21:24:23 dist-test-slave-4d5t krb5kdc[25225](info): commencing operation
krb5kdc: starting...
W20260926 21:24:24.510407 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:24:24 dist-test-slave-4d5t krb5kdc[25225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:24:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:24:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:24:24.778606 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:39893
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:39893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:24:25.132177 25238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:25.132561 25238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:25.132748 25238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:25.140487 25238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:24:25.140697 25238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:25.140805 25238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:25.140906 25238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:24:25.140986 25238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:24:25.151464 25238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:39893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:25.152954 25238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:25.154698 25238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:25.169054 25243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:25.169027 25246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:25.169094 25244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:25.169888 25238 server_base.cc:1034] running on GCE node
I20260926 21:24:25.170845 25238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:25.172485 25238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:25.173806 25238 hybrid_clock.cc:648] HybridClock initialized: now 1790457865173744 us; error 49 us; skew 500 ppm
I20260926 21:24:25.174191 25238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:25 dist-test-slave-4d5t krb5kdc[25225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457865, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:25.177793 25238 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:24:25.179394 25238 webserver.cc:460] Webserver started at http://127.14.142.62:35763/ using document root <none> and password file <none>
I20260926 21:24:25.180158 25238 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:25.180354 25238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:25.180763 25238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:25.183992 25238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "c30998fcc57a4d239477a392d6ed534b"
format_stamp: "Formatted at 2026-09-26 21:24:25 on dist-test-slave-4d5t"
server_key: "652d402b8d4fc885f14757281717d731"
server_key_iv: "432c2c4abf9a177c03472cdee9157747"
server_key_version: "encryptionkey@0"
I20260926 21:24:25.184911 25238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "c30998fcc57a4d239477a392d6ed534b"
format_stamp: "Formatted at 2026-09-26 21:24:25 on dist-test-slave-4d5t"
server_key: "652d402b8d4fc885f14757281717d731"
server_key_iv: "432c2c4abf9a177c03472cdee9157747"
server_key_version: "encryptionkey@0"
I20260926 21:24:25.191572 25238 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:24:25.195478 25253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:25.197124 25238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:24:25.197337 25238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "c30998fcc57a4d239477a392d6ed534b"
format_stamp: "Formatted at 2026-09-26 21:24:25 on dist-test-slave-4d5t"
server_key: "652d402b8d4fc885f14757281717d731"
server_key_iv: "432c2c4abf9a177c03472cdee9157747"
server_key_version: "encryptionkey@0"
I20260926 21:24:25.197566 25238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:25.227300 25238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:25.232366 25238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:25.232734 25238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:25.254051 25238 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:39893
I20260926 21:24:25.254074 25304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:39893 every 8 connection(s)
I20260926 21:24:25.255569 25238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:24:25.260243 25305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:24:25.265878 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 25238
I20260926 21:24:25.266255 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:24:25.267237 14904 external_mini_cluster.cc:1428] Setting key 4f076a01a765e2afdb6d7d023d3dfd1b
I20260926 21:24:25.273471 25305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b: Bootstrap starting.
I20260926 21:24:25.277446 25305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b: Neither blocks nor log segments found. Creating new log.
I20260926 21:24:25.278543 25305 log.cc:824] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:25.282051 25305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b: No bootstrap required, opened a new log
Sep 26 21:24:25 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:24:25.291898 25305 raft_consensus.cc:348] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30998fcc57a4d239477a392d6ed534b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 39893 } }
I20260926 21:24:25.292435 25305 raft_consensus.cc:374] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:24:25.292586 25305 raft_consensus.cc:729] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30998fcc57a4d239477a392d6ed534b, State: Initialized, Role: FOLLOWER
I20260926 21:24:25.293310 25305 consensus_queue.cc:260] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30998fcc57a4d239477a392d6ed534b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 39893 } }
I20260926 21:24:25.293608 25305 raft_consensus.cc:388] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:24:25.293800 25305 raft_consensus.cc:482] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:24:25.294013 25305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:24:25.295322 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:25.270239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.14.142.62:39893)
0926 21:24:25.271100 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:25.271112 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:25.271157 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:24:25.273521 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:25.273547 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:25.274051 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:25.274441 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:25.276840 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:25.278467 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:25.280164 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:25.281002 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:25.281151 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:25.286360 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:25.286369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:25.286386 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:25.286432 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:25.290646 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:25.291592 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:25.291600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:25.291607 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:25.291790 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:25.292220 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:25.292224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:25.292226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:25.293018 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:25.293277 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:25.293766 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 21:24:25.294252 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":418,"threads_started":1}
I20260926 21:24:25.296644 25305 raft_consensus.cc:504] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30998fcc57a4d239477a392d6ed534b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 39893 } }
I20260926 21:24:25.297252 25305 leader_election.cc:302] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c30998fcc57a4d239477a392d6ed534b; no voters:
I20260926 21:24:25.297952 25305 leader_election.cc:288] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:24:25.298077 25310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:24:25.298830 25310 raft_consensus.cc:686] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [term 1 LEADER]: Becoming Leader. State: Replica: c30998fcc57a4d239477a392d6ed534b, State: Running, Role: LEADER
I20260926 21:24:25.299327 25310 consensus_queue.cc:237] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30998fcc57a4d239477a392d6ed534b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 39893 } }
I20260926 21:24:25.301080 25310 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c30998fcc57a4d239477a392d6ed534b. Latest consensus state: current_term: 1 leader_uuid: "c30998fcc57a4d239477a392d6ed534b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30998fcc57a4d239477a392d6ed534b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 39893 } } }
I20260926 21:24:25.301424 25310 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:25.303325 25312 sys_catalog.cc:455] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c30998fcc57a4d239477a392d6ed534b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c30998fcc57a4d239477a392d6ed534b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 39893 } } }
I20260926 21:24:25.303820 25312 sys_catalog.cc:458] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [sys.catalog]: This master's current role is: LEADER
I20260926 21:24:25.306309 25305 sys_catalog.cc:564] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:24:25.306586 25314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:24:25.316874 25314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:24:25.327980 25314 catalog_manager.cc:1347] Generated new cluster ID: 6b636d29fb2d481fa8f815e93981e971
I20260926 21:24:25.328137 25314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:24:25.349018 25314 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:24:25.351183 25314 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:24:25.364845 25314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b: Generated new TSK 0
I20260926 21:24:25.365823 25314 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:24:25.446779 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:39893
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:24:25.798672 25331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:25.798964 25331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:25.799062 25331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:25.808039 25331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:25.808228 25331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:25.808538 25331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:24:25.821863 25331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:39893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:25.823226 25331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:25.824668 25331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:25.838702 25337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:25.838793 25336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:25.838863 25339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:25.839991 25331 server_base.cc:1034] running on GCE node
I20260926 21:24:25.841071 25331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:25.842149 25331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:25.843461 25331 hybrid_clock.cc:648] HybridClock initialized: now 1790457865843393 us; error 61 us; skew 500 ppm
I20260926 21:24:25.843851 25331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:25 dist-test-slave-4d5t krb5kdc[25225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457865, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:25.847476 25331 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:24:25.849316 25331 webserver.cc:460] Webserver started at http://127.14.142.1:42485/ using document root <none> and password file <none>
I20260926 21:24:25.850059 25331 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:25.850214 25331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:25.850533 25331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:25.853873 25331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "72b871b616dc4c0ca0d5fb296be9cdf6"
format_stamp: "Formatted at 2026-09-26 21:24:25 on dist-test-slave-4d5t"
server_key: "b7afc4586b35ff542c4c6e54c0e3173c"
server_key_iv: "f140bd4b9c6be0b1825598d57cdcede1"
server_key_version: "encryptionkey@0"
I20260926 21:24:25.854761 25331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "72b871b616dc4c0ca0d5fb296be9cdf6"
format_stamp: "Formatted at 2026-09-26 21:24:25 on dist-test-slave-4d5t"
server_key: "b7afc4586b35ff542c4c6e54c0e3173c"
server_key_iv: "f140bd4b9c6be0b1825598d57cdcede1"
server_key_version: "encryptionkey@0"
I20260926 21:24:25.860436 25331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:24:25.864266 25346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:25.866441 25331 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:24:25.866657 25331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "72b871b616dc4c0ca0d5fb296be9cdf6"
format_stamp: "Formatted at 2026-09-26 21:24:25 on dist-test-slave-4d5t"
server_key: "b7afc4586b35ff542c4c6e54c0e3173c"
server_key_iv: "f140bd4b9c6be0b1825598d57cdcede1"
server_key_version: "encryptionkey@0"
I20260926 21:24:25.866878 25331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:25.886196 25331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:25.891659 25331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:25.892058 25331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:25.893329 25331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:25.894837 25331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:25.894953 25331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:25.895134 25331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:25.895211 25331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:25.928833 25331 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:39231
I20260926 21:24:25.928974 25458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:39231 every 8 connection(s)
I20260926 21:24:25.930197 25331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:24:25.937168 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 25331
I20260926 21:24:25.937378 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:24:25.938006 14904 external_mini_cluster.cc:1428] Setting key 9d85ee72411fd57e0666447eeac93d16
Sep 26 21:24:25 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457865, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:24:25.952605 25461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:25.933368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51089 (local address 127.14.142.62:39893)
0926 21:24:25.933866 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:25.933871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:25.934809 (+ 938us) server_negotiation.cc:407] Connection header received
0926 21:24:25.936713 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:25.936721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:25.936819 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:25.936960 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:25.938711 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:25.939598 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:25.941486 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:25.942257 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:25.942415 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:25.946180 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:25.946189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:25.946196 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:25.946245 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:25.949393 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:25.950160 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:25.950169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:25.950173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:25.950305 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:25.950749 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:25.950756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:25.950759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:25.951056 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:25.951228 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:25.951966 (+ 738us) server_negotiation.cc:299] Negotiation successful
0926 21:24:25.952208 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":262,"threads_started":1}
I20260926 21:24:25.953624 25462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:25.933906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:39893 (local address 127.14.142.1:51089)
0926 21:24:25.934631 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:25.934679 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:25.936332 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:25.937215 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:25.937223 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:25.937858 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:25.938481 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:25.938499 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:25.939783 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:25.939788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:25.941318 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:25.941331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:25.942551 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:25.942557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:25.942769 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:25.943478 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:25.943506 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:25.945907 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:25.949556 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:24:25.949576 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:24:25.949591 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:24:25.950027 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:24:25.950424 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:24:25.950429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:24:25.950434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:24:25.950633 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:24:25.951203 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:25.951211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:25.951596 (+ 385us) client_negotiation.cc:769] Sending connection context
0926 21:24:25.951875 (+ 279us) client_negotiation.cc:240] Negotiation successful
0926 21:24:25.952430 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":291,"threads_started":1}
I20260926 21:24:25.955607 25459 heartbeater.cc:347] Connected to a master server at 127.14.142.62:39893
I20260926 21:24:25.956159 25459 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:25.957031 25459 heartbeater.cc:511] Master 127.14.142.62:39893 requested a full tablet report, sending...
I20260926 21:24:25.959398 25270 ts_manager.cc:194] Registered new tserver with Master: 72b871b616dc4c0ca0d5fb296be9cdf6 (127.14.142.1:39231)
I20260926 21:24:25.960947 25270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:51089
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:24:25.996899 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:39893
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:24:26.346827 25465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:26.347198 25465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:26.347381 25465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:26.354962 25465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:26.355144 25465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:26.355330 25465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:24:26.365939 25465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:39893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:26.367306 25465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:26.368698 25465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:26.382428 25470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:26.382360 25473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:26.382360 25471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:26.383947 25465 server_base.cc:1034] running on GCE node
I20260926 21:24:26.385023 25465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:26.386471 25465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:26.387706 25465 hybrid_clock.cc:648] HybridClock initialized: now 1790457866387638 us; error 62 us; skew 500 ppm
I20260926 21:24:26.388043 25465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:26 dist-test-slave-4d5t krb5kdc[25225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457866, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:26.391498 25465 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:24:26.393237 25465 webserver.cc:460] Webserver started at http://127.14.142.2:36243/ using document root <none> and password file <none>
I20260926 21:24:26.393959 25465 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:26.394097 25465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:26.394418 25465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:26.397493 25465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "64532c56121844a9a047d5dba30218c4"
format_stamp: "Formatted at 2026-09-26 21:24:26 on dist-test-slave-4d5t"
server_key: "d49fdaf752f3a4015888ae308f405de9"
server_key_iv: "ad1258d69ea8db896456bf4b17e80768"
server_key_version: "encryptionkey@0"
I20260926 21:24:26.398252 25465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "64532c56121844a9a047d5dba30218c4"
format_stamp: "Formatted at 2026-09-26 21:24:26 on dist-test-slave-4d5t"
server_key: "d49fdaf752f3a4015888ae308f405de9"
server_key_iv: "ad1258d69ea8db896456bf4b17e80768"
server_key_version: "encryptionkey@0"
I20260926 21:24:26.403609 25465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 21:24:26.407788 25480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:26.409210 25465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:24:26.409381 25465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "64532c56121844a9a047d5dba30218c4"
format_stamp: "Formatted at 2026-09-26 21:24:26 on dist-test-slave-4d5t"
server_key: "d49fdaf752f3a4015888ae308f405de9"
server_key_iv: "ad1258d69ea8db896456bf4b17e80768"
server_key_version: "encryptionkey@0"
I20260926 21:24:26.409587 25465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:26.431497 25465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:26.437791 25465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:26.438140 25465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:26.439280 25465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:26.440794 25465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:26.440873 25465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:26.441023 25465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:26.441072 25465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:26.477970 25465 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:45631
I20260926 21:24:26.478061 25592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:45631 every 8 connection(s)
I20260926 21:24:26.479349 25465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:24:26.484015 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 25465
I20260926 21:24:26.484198 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:24:26.484745 14904 external_mini_cluster.cc:1428] Setting key feb5f0dd78d98e2b72a2841aa56a77c3
Sep 26 21:24:26 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457866, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:24:26.503664 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:26.482073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:48717 (local address 127.14.142.62:39893)
0926 21:24:26.482527 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:26.482531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:26.483387 (+ 856us) server_negotiation.cc:407] Connection header received
0926 21:24:26.485470 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:26.485475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:26.485581 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:26.485704 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:26.487203 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:26.488036 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:26.489708 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:26.490135 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:26.490273 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:26.494998 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:26.495008 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:26.495016 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:26.495075 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:26.499071 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:26.500033 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:26.500042 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:26.500049 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:26.500222 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:26.501293 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:26.501302 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:26.501308 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:26.501763 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:26.501919 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:26.502782 (+ 863us) server_negotiation.cc:299] Negotiation successful
0926 21:24:26.503176 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":251,"threads_started":1}
I20260926 21:24:26.504472 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:26.482528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:39893 (local address 127.14.142.2:48717)
0926 21:24:26.483218 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:26.483261 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:26.485189 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:26.485940 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:26.485948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:26.486467 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:26.487029 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:26.487050 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:26.488204 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:26.488208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:26.489545 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:26.489566 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:26.490364 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:26.490371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:26.490624 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:26.491602 (+ 978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:26.491638 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:26.494730 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:26.499319 (+ 4589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:24:26.499328 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:24:26.499352 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:24:26.499864 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:24:26.500549 (+ 685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:24:26.500561 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:24:26.500569 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:24:26.501142 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:24:26.501930 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:26.501938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:26.502358 (+ 420us) client_negotiation.cc:769] Sending connection context
0926 21:24:26.502754 (+ 396us) client_negotiation.cc:240] Negotiation successful
0926 21:24:26.503253 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":268,"threads_started":1}
I20260926 21:24:26.506512 25593 heartbeater.cc:347] Connected to a master server at 127.14.142.62:39893
I20260926 21:24:26.506904 25593 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:26.507732 25593 heartbeater.cc:511] Master 127.14.142.62:39893 requested a full tablet report, sending...
I20260926 21:24:26.509413 25270 ts_manager.cc:194] Registered new tserver with Master: 64532c56121844a9a047d5dba30218c4 (127.14.142.2:45631)
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
I20260926 21:24:26.510332 25270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:48717
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:24:26.545747 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:39893
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:24:26.896399 25599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:24:26.896847 25599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:24:26.897011 25599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:24:26.905103 25599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:24:26.905396 25599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:24:26.905756 25599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:24:26.918596 25599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:39893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:24:26.920194 25599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:24:26.921581 25599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:24:26.936136 25605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:26.936131 25607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:24:26.936177 25604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:24:26.936980 25599 server_base.cc:1034] running on GCE node
I20260926 21:24:26.937595 25599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:24:26.938562 25599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:24:26.939808 25599 hybrid_clock.cc:648] HybridClock initialized: now 1790457866939760 us; error 39 us; skew 500 ppm
I20260926 21:24:26.940168 25599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:24:26 dist-test-slave-4d5t krb5kdc[25225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457866, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:24:26.943707 25599 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:24:26.945663 25599 webserver.cc:460] Webserver started at http://127.14.142.3:40437/ using document root <none> and password file <none>
I20260926 21:24:26.946575 25599 fs_manager.cc:359] Metadata directory not provided
I20260926 21:24:26.946760 25599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:24:26.947098 25599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:24:26.950296 25599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "e6437a898c3b4756a6a623a57634a0ef"
format_stamp: "Formatted at 2026-09-26 21:24:26 on dist-test-slave-4d5t"
server_key: "19bccd9c395f5528bb553ca1d16fdc73"
server_key_iv: "af9d6bc3b909dde0f1e22cb74c06061d"
server_key_version: "encryptionkey@0"
I20260926 21:24:26.951109 25599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "e6437a898c3b4756a6a623a57634a0ef"
format_stamp: "Formatted at 2026-09-26 21:24:26 on dist-test-slave-4d5t"
server_key: "19bccd9c395f5528bb553ca1d16fdc73"
server_key_iv: "af9d6bc3b909dde0f1e22cb74c06061d"
server_key_version: "encryptionkey@0"
I20260926 21:24:26.956768 25599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:24:26.960884 25614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:26.962666 25599 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:24:26.962891 25599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "e6437a898c3b4756a6a623a57634a0ef"
format_stamp: "Formatted at 2026-09-26 21:24:26 on dist-test-slave-4d5t"
server_key: "19bccd9c395f5528bb553ca1d16fdc73"
server_key_iv: "af9d6bc3b909dde0f1e22cb74c06061d"
server_key_version: "encryptionkey@0"
I20260926 21:24:26.963114 25599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:24:26.964293 25459 heartbeater.cc:503] Master 127.14.142.62:39893 was elected leader, sending a full tablet report...
I20260926 21:24:26.990762 25599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:24:26.995889 25599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:24:26.996176 25599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:24:26.997169 25599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:24:26.998504 25599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:24:26.998579 25599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:26.998664 25599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:24:26.998708 25599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:24:27.037598 25599 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:42263
I20260926 21:24:27.037704 25726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:42263 every 8 connection(s)
I20260926 21:24:27.038923 25599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:24:27.042711 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 25599
I20260926 21:24:27.042956 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:24:27.043553 14904 external_mini_cluster.cc:1428] Setting key 3396e7b613757f02917f168bfb45f659
Sep 26 21:24:27 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457866, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:24:27.066604 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.041709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:53355 (local address 127.14.142.62:39893)
0926 21:24:27.042193 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.042199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.043708 (+ 1509us) server_negotiation.cc:407] Connection header received
0926 21:24:27.045627 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.045633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.045911 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.046168 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:27.048677 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.049593 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.053074 (+ 3481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.053918 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.054134 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.059149 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:27.059157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:27.059161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:27.059207 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:27.062685 (+ 3478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.063849 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.063854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.063858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.064050 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.064640 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.064667 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.064671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.065026 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:27.065177 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.066114 (+ 937us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.066300 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":218,"threads_started":1}
I20260926 21:24:27.068456 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.042378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:39893 (local address 127.14.142.3:53355)
0926 21:24:27.043352 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:27.043393 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:27.045341 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:27.046680 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:27.046697 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:27.047520 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:24:27.048344 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.048371 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.049801 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.049806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.052715 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.052751 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.054282 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.054293 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.054673 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.055571 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:24:27.055597 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:24:27.058830 (+ 3233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:24:27.062879 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:24:27.062890 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:24:27.062910 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:24:27.063572 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:24:27.064203 (+ 631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:24:27.064207 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:24:27.064211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:24:27.064484 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:24:27.065166 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:24:27.065177 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:24:27.065613 (+ 436us) client_negotiation.cc:769] Sending connection context
0926 21:24:27.065984 (+ 371us) client_negotiation.cc:240] Negotiation successful
0926 21:24:27.066582 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":438,"threads_started":1}
I20260926 21:24:27.070344 25727 heartbeater.cc:347] Connected to a master server at 127.14.142.62:39893
I20260926 21:24:27.070741 25727 heartbeater.cc:464] Registering TS with master...
I20260926 21:24:27.071695 25727 heartbeater.cc:511] Master 127.14.142.62:39893 requested a full tablet report, sending...
I20260926 21:24:27.073350 25270 ts_manager.cc:194] Registered new tserver with Master: e6437a898c3b4756a6a623a57634a0ef (127.14.142.3:42263)
I20260926 21:24:27.074134 25270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:53355
I20260926 21:24:27.079895 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:24:27.099957 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.084704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.14.142.62:39893)
0926 21:24:27.084894 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.084899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.085285 (+ 386us) server_negotiation.cc:407] Connection header received
0926 21:24:27.085691 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.085697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.085783 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.085888 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:27.087419 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.088163 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.089596 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.090170 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.090352 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.093576 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:27.093583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:27.093587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:27.093640 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:27.097225 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.098036 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.098041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.098043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.098197 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.098604 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.098607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.098609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.098946 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:27.099206 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.099532 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.099720 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 21:24:27.104773 25270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:24:27.112746 25270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:24:27.145473 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.130424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.14.142.1:39231)
0926 21:24:27.133120 (+ 2696us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.133132 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.133168 (+ 36us) server_negotiation.cc:407] Connection header received
0926 21:24:27.133296 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.133304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.133589 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.134702 (+ 1113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:24:27.135032 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.136781 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.141810 (+ 5029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.143454 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.144503 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.144792 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.144899 (+ 107us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.145082 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2506,"thread_start_us":2349,"threads_started":1}
I20260926 21:24:27.145538 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.130177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39231 (local address 127.0.0.1:42552)
0926 21:24:27.131696 (+ 1519us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:27.131753 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:27.131978 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:27.133875 (+ 1897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:27.133885 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:27.133925 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:24:27.134479 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.134492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.138534 (+ 4042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.138540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.141521 (+ 2981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.141541 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.144321 (+ 2780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.144329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.144636 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.144689 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 21:24:27.144854 (+ 165us) client_negotiation.cc:240] Negotiation successful
0926 21:24:27.145094 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":45,"thread_start_us":410,"threads_started":1}
I20260926 21:24:27.151221 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.133218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:42263 (local address 127.0.0.1:41840)
0926 21:24:27.134036 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:27.134065 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:27.134178 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:27.136818 (+ 2640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:27.136831 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:27.136874 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:24:27.137596 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.137614 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.142026 (+ 4412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.142035 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.144991 (+ 2956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.145008 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.149906 (+ 4898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.149915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.150125 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.150459 (+ 334us) client_negotiation.cc:769] Sending connection context
0926 21:24:27.150556 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 21:24:27.150702 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":249,"thread_start_us":489,"threads_started":1}
I20260926 21:24:27.152189 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.132861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.14.142.3:42263)
0926 21:24:27.134879 (+ 2018us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.134891 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.134934 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:24:27.135069 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.135079 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.135344 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.135574 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:24:27.139471 (+ 3897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.141796 (+ 2325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.145197 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.147399 (+ 2202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.147570 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.147887 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.151557 (+ 3670us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.151794 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1798,"thread_start_us":2498,"threads_started":1}
I20260926 21:24:27.152048 25394 tablet_service.cc:1467] Processing CreateTablet for tablet fb8c8a6ca54648058e31b549c64bc108 (DEFAULT_TABLE table=test-table [id=879d5e4f7de54634baf8f3a8ad31a4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:24:27.153806 25394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb8c8a6ca54648058e31b549c64bc108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:24:27.155584 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.137051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:45631 (local address 127.0.0.1:55772)
0926 21:24:27.145777 (+ 8726us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:27.145797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:27.145917 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:27.146647 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:27.146652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:27.146677 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:24:27.146847 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.146857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.150145 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.150151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.152327 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.152346 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.154735 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.154741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.154954 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.154992 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:24:27.155066 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 21:24:27.155215 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8631,"mutex_wait_us":25,"thread_start_us":11019,"threads_started":1}
I20260926 21:24:27.155738 25662 tablet_service.cc:1467] Processing CreateTablet for tablet fb8c8a6ca54648058e31b549c64bc108 (DEFAULT_TABLE table=test-table [id=879d5e4f7de54634baf8f3a8ad31a4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:24:27.156909 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.137984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.14.142.2:45631)
0926 21:24:27.138602 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.138610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.146054 (+ 7444us) server_negotiation.cc:407] Connection header received
0926 21:24:27.146177 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.146187 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.146439 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.146677 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:24:27.147022 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.149958 (+ 2936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.152942 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.154513 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.156009 (+ 1496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.156230 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.156301 (+ 71us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.156530 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":312,"threads_started":1}
I20260926 21:24:27.158108 25662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb8c8a6ca54648058e31b549c64bc108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:24:27.160709 25528 tablet_service.cc:1467] Processing CreateTablet for tablet fb8c8a6ca54648058e31b549c64bc108 (DEFAULT_TABLE table=test-table [id=879d5e4f7de54634baf8f3a8ad31a4dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:24:27.162117 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb8c8a6ca54648058e31b549c64bc108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:24:27.170802 25746 tablet_bootstrap.cc:492] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Bootstrap starting.
I20260926 21:24:27.176139 25746 tablet_bootstrap.cc:654] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Neither blocks nor log segments found. Creating new log.
I20260926 21:24:27.176709 25748 tablet_bootstrap.cc:492] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Bootstrap starting.
I20260926 21:24:27.177698 25747 tablet_bootstrap.cc:492] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Bootstrap starting.
I20260926 21:24:27.178802 25746 log.cc:824] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:27.180698 25748 tablet_bootstrap.cc:654] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Neither blocks nor log segments found. Creating new log.
I20260926 21:24:27.181339 25747 tablet_bootstrap.cc:654] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Neither blocks nor log segments found. Creating new log.
I20260926 21:24:27.181905 25746 tablet_bootstrap.cc:492] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: No bootstrap required, opened a new log
I20260926 21:24:27.182056 25748 log.cc:824] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:27.182189 25746 ts_tablet_manager.cc:1397] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260926 21:24:27.182554 25747 log.cc:824] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Log is configured to *not* fsync() on all Append() calls
I20260926 21:24:27.189320 25747 tablet_bootstrap.cc:492] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: No bootstrap required, opened a new log
I20260926 21:24:27.189659 25747 ts_tablet_manager.cc:1397] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260926 21:24:27.192137 25748 tablet_bootstrap.cc:492] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: No bootstrap required, opened a new log
I20260926 21:24:27.192468 25748 ts_tablet_manager.cc:1397] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260926 21:24:27.193507 25746 raft_consensus.cc:348] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.194028 25746 raft_consensus.cc:374] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:24:27.194166 25746 raft_consensus.cc:729] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b871b616dc4c0ca0d5fb296be9cdf6, State: Initialized, Role: FOLLOWER
I20260926 21:24:27.195030 25746 consensus_queue.cc:260] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.197077 25746 ts_tablet_manager.cc:1428] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260926 21:24:27.199673 25747 raft_consensus.cc:348] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.200110 25747 raft_consensus.cc:374] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:24:27.200209 25747 raft_consensus.cc:729] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6437a898c3b4756a6a623a57634a0ef, State: Initialized, Role: FOLLOWER
I20260926 21:24:27.201054 25747 consensus_queue.cc:260] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.202247 25727 heartbeater.cc:503] Master 127.14.142.62:39893 was elected leader, sending a full tablet report...
I20260926 21:24:27.202980 25747 ts_tablet_manager.cc:1428] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260926 21:24:27.204970 25748 raft_consensus.cc:348] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.205386 25748 raft_consensus.cc:374] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:24:27.205592 25748 raft_consensus.cc:729] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64532c56121844a9a047d5dba30218c4, State: Initialized, Role: FOLLOWER
I20260926 21:24:27.206297 25748 consensus_queue.cc:260] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.207584 25593 heartbeater.cc:503] Master 127.14.142.62:39893 was elected leader, sending a full tablet report...
I20260926 21:24:27.208217 25748 ts_tablet_manager.cc:1428] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
W20260926 21:24:27.231550 25594 tablet.cc:2369] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:24:27.290951 25728 tablet.cc:2369] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:24:27.432916 25460 tablet.cc:2369] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:24:27.493736 25753 raft_consensus.cc:482] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:24:27.494091 25753 raft_consensus.cc:504] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.495961 25753 leader_election.cc:288] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72b871b616dc4c0ca0d5fb296be9cdf6 (127.14.142.1:39231), 64532c56121844a9a047d5dba30218c4 (127.14.142.2:45631)
I20260926 21:24:27.503690 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.496531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:35575 (local address 127.14.142.1:39231)
0926 21:24:27.496809 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.496817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.496846 (+ 29us) server_negotiation.cc:407] Connection header received
0926 21:24:27.497053 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.497061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.497160 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.497300 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:24:27.497737 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.498873 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.501132 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.502252 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.502375 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.502428 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.503258 (+ 830us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.503391 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260926 21:24:27.503809 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.496277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39231 (local address 127.14.142.3:35575)
0926 21:24:27.496638 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:27.496666 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:27.496832 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:27.497310 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:27.497314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:27.497340 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:24:27.497547 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.497556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.499039 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.499045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.500963 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.500979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.502551 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.502560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.502928 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.502989 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 21:24:27.503169 (+ 180us) client_negotiation.cc:240] Negotiation successful
0926 21:24:27.503360 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":1}
I20260926 21:24:27.504683 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.496673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:43135 (local address 127.14.142.2:45631)
0926 21:24:27.496892 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.496902 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.497469 (+ 567us) server_negotiation.cc:407] Connection header received
0926 21:24:27.497681 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.497689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.497760 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.497828 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:24:27.498559 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.499483 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.501808 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.503403 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.503604 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.503689 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.504190 (+ 501us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.504370 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 21:24:27.504798 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.496332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:45631 (local address 127.14.142.3:43135)
0926 21:24:27.497342 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0926 21:24:27.497364 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:24:27.497536 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:24:27.497894 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:24:27.497900 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:24:27.497930 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:24:27.498350 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.498364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.499679 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.499684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.501646 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:24:27.501660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.503710 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.503735 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:24:27.503959 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.504018 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 21:24:27.504147 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 21:24:27.504301 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":1,"thread_start_us":660,"threads_started":1}
I20260926 21:24:27.505434 25414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb8c8a6ca54648058e31b549c64bc108" candidate_uuid: "e6437a898c3b4756a6a623a57634a0ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" is_pre_election: true
I20260926 21:24:27.506125 25414 raft_consensus.cc:2454] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6437a898c3b4756a6a623a57634a0ef in term 0.
I20260926 21:24:27.506259 25548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb8c8a6ca54648058e31b549c64bc108" candidate_uuid: "e6437a898c3b4756a6a623a57634a0ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64532c56121844a9a047d5dba30218c4" is_pre_election: true
I20260926 21:24:27.507148 25548 raft_consensus.cc:2454] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6437a898c3b4756a6a623a57634a0ef in term 0.
I20260926 21:24:27.507120 25618 leader_election.cc:302] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72b871b616dc4c0ca0d5fb296be9cdf6, e6437a898c3b4756a6a623a57634a0ef; no voters:
I20260926 21:24:27.507771 25753 raft_consensus.cc:2781] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:24:27.508033 25753 raft_consensus.cc:482] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:24:27.508162 25753 raft_consensus.cc:3037] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:24:27.511076 25753 raft_consensus.cc:504] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.511821 25753 leader_election.cc:288] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [CANDIDATE]: Term 1 election: Requested vote from peers 72b871b616dc4c0ca0d5fb296be9cdf6 (127.14.142.1:39231), 64532c56121844a9a047d5dba30218c4 (127.14.142.2:45631)
I20260926 21:24:27.512615 25414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb8c8a6ca54648058e31b549c64bc108" candidate_uuid: "e6437a898c3b4756a6a623a57634a0ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6"
I20260926 21:24:27.512688 25548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb8c8a6ca54648058e31b549c64bc108" candidate_uuid: "e6437a898c3b4756a6a623a57634a0ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64532c56121844a9a047d5dba30218c4"
I20260926 21:24:27.512884 25414 raft_consensus.cc:3037] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:24:27.513000 25548 raft_consensus.cc:3037] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:24:27.515617 25414 raft_consensus.cc:2454] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6437a898c3b4756a6a623a57634a0ef in term 1.
I20260926 21:24:27.516121 25548 raft_consensus.cc:2454] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6437a898c3b4756a6a623a57634a0ef in term 1.
I20260926 21:24:27.516285 25618 leader_election.cc:302] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72b871b616dc4c0ca0d5fb296be9cdf6, e6437a898c3b4756a6a623a57634a0ef; no voters:
I20260926 21:24:27.516868 25753 raft_consensus.cc:2781] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:24:27.517558 25753 raft_consensus.cc:686] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 1 LEADER]: Becoming Leader. State: Replica: e6437a898c3b4756a6a623a57634a0ef, State: Running, Role: LEADER
I20260926 21:24:27.518302 25753 consensus_queue.cc:237] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } }
I20260926 21:24:27.523131 25270 catalog_manager.cc:5495] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef reported cstate change: term changed from 0 to 1, leader changed from <none> to e6437a898c3b4756a6a623a57634a0ef (127.14.142.3). New cstate: current_term: 1 leader_uuid: "e6437a898c3b4756a6a623a57634a0ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6437a898c3b4756a6a623a57634a0ef" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 } health_report { overall_health: UNKNOWN } } }
I20260926 21:24:27.605504 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.598512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.14.142.3:42263)
0926 21:24:27.598733 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.598738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.598835 (+ 97us) server_negotiation.cc:407] Connection header received
0926 21:24:27.599040 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.599044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.599137 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.599301 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:24:27.599940 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.601028 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.602733 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.603143 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.603267 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.603872 (+ 605us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:24:27.604824 (+ 952us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:24:27.604985 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.605185 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.605273 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 21:24:27.614257 25414 raft_consensus.cc:1261] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 1 FOLLOWER]: Refusing update from remote peer e6437a898c3b4756a6a623a57634a0ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:24:27.614274 25548 raft_consensus.cc:1261] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 1 FOLLOWER]: Refusing update from remote peer e6437a898c3b4756a6a623a57634a0ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:24:27.615360 25756 consensus_queue.cc:1035] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "64532c56121844a9a047d5dba30218c4" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 45631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:24:27.615712 25753 consensus_queue.cc:1035] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "72b871b616dc4c0ca0d5fb296be9cdf6" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:24:27.646612 25270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:24:27.646904 25270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:24:27.651012 25270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c30998fcc57a4d239477a392d6ed534b: Sending DeleteTablet for 3 replicas of tablet fb8c8a6ca54648058e31b549c64bc108
I20260926 21:24:27.652525 25394 tablet_service.cc:1514] Processing DeleteTablet for tablet fb8c8a6ca54648058e31b549c64bc108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:24:27 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:42552
I20260926 21:24:27.652824 25528 tablet_service.cc:1514] Processing DeleteTablet for tablet fb8c8a6ca54648058e31b549c64bc108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:24:27 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:55772
I20260926 21:24:27.652878 25662 tablet_service.cc:1514] Processing DeleteTablet for tablet fb8c8a6ca54648058e31b549c64bc108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:24:27 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41840
I20260926 21:24:27.656194 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 25331
I20260926 21:24:27.665640 25770 tablet_replica.cc:330] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: stopping tablet replica
I20260926 21:24:27.666484 25770 raft_consensus.cc:2229] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:24:27.667322 25770 raft_consensus.cc:2258] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:24:27.670936 25771 tablet_replica.cc:330] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: stopping tablet replica
I20260926 21:24:27.671787 25770 ts_tablet_manager.cc:1905] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:24:27.672653 25771 raft_consensus.cc:2229] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:24:27.673584 25771 raft_consensus.cc:2258] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:24:27.676175 25771 ts_tablet_manager.cc:1905] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:24:27.677130 25769 tablet_replica.cc:330] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: stopping tablet replica
I20260926 21:24:27.677606 25769 raft_consensus.cc:2229] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:24:27.678180 25769 raft_consensus.cc:2258] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:24:27.679860 25770 ts_tablet_manager.cc:1918] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:24:27.680054 25770 log.cc:1196] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/fb8c8a6ca54648058e31b549c64bc108
I20260926 21:24:27.680465 25769 ts_tablet_manager.cc:1905] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:24:27.680725 25770 ts_tablet_manager.cc:1939] T fb8c8a6ca54648058e31b549c64bc108 P 64532c56121844a9a047d5dba30218c4: Deleting consensus metadata
I20260926 21:24:27.682928 25771 ts_tablet_manager.cc:1918] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:24:27.682898 25256 catalog_manager.cc:4841] TS 64532c56121844a9a047d5dba30218c4 (127.14.142.2:45631): tablet fb8c8a6ca54648058e31b549c64bc108 (table test-table [id=879d5e4f7de54634baf8f3a8ad31a4dc]) successfully deleted
I20260926 21:24:27.683096 25771 log.cc:1196] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/fb8c8a6ca54648058e31b549c64bc108
I20260926 21:24:27.683662 25771 ts_tablet_manager.cc:1939] T fb8c8a6ca54648058e31b549c64bc108 P e6437a898c3b4756a6a623a57634a0ef: Deleting consensus metadata
I20260926 21:24:27.685577 25255 catalog_manager.cc:4841] TS e6437a898c3b4756a6a623a57634a0ef (127.14.142.3:42263): tablet fb8c8a6ca54648058e31b549c64bc108 (table test-table [id=879d5e4f7de54634baf8f3a8ad31a4dc]) successfully deleted
I20260926 21:24:27.688482 25769 ts_tablet_manager.cc:1918] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:24:27.688725 25769 log.cc:1196] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/fb8c8a6ca54648058e31b549c64bc108
I20260926 21:24:27.689322 25769 ts_tablet_manager.cc:1939] T fb8c8a6ca54648058e31b549c64bc108 P 72b871b616dc4c0ca0d5fb296be9cdf6: Deleting consensus metadata
Sep 26 21:24:27 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:24:27.691514 25257 catalog_manager.cc:4841] TS 72b871b616dc4c0ca0d5fb296be9cdf6 (127.14.142.1:39231): tablet fb8c8a6ca54648058e31b549c64bc108 (table test-table [id=879d5e4f7de54634baf8f3a8ad31a4dc]) successfully deleted
I20260926 21:24:27.696777 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.660213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.14.142.1:39231)
0926 21:24:27.660457 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.660467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.671246 (+ 10779us) server_negotiation.cc:407] Connection header received
0926 21:24:27.671476 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.671484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.671591 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.671697 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:27.673297 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.674321 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.685476 (+ 11155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.686189 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.686363 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.690612 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:27.690623 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:27.690637 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:27.690678 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:27.693840 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.694557 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.694565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.694567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.694697 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.695090 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.695094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.695096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.695469 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:27.695785 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.696110 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.696349 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:24:27.697469 25454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:27.810053 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 25331
I20260926 21:24:27.828838 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 25465
Sep 26 21:24:27 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:24:27.848982 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.829760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.14.142.2:45631)
0926 21:24:27.829981 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.829988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.830013 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:24:27.830139 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.830146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.830237 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.830372 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:27.831690 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.833112 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.834798 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.835512 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.835683 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.840750 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:27.840771 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:27.840796 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:27.840850 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:27.844932 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.846225 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.846236 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.846240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.846428 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.846970 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.846974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.846977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.847610 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:27.848054 (+ 444us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.848297 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.848546 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:24:27.849977 25588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:27.945469 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 25465
I20260926 21:24:27.970288 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 25599
Sep 26 21:24:27 dist-test-slave-4d5t krb5kdc[25225](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:24:27.990868 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:24:27.971107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41866 (local address 127.14.142.3:42263)
0926 21:24:27.971304 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 21:24:27.971315 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:24:27.971345 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:24:27.971491 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:24:27.971497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:24:27.971615 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:24:27.971766 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:24:27.973068 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.974247 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.975804 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:24:27.976496 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:24:27.976677 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:24:27.981679 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:24:27.981706 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:24:27.981735 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:24:27.981806 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:24:27.987437 (+ 5631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.988322 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.988331 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.988335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.988592 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:24:27.988989 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:24:27.988994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:24:27.988998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:24:27.989479 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:24:27.989791 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 21:24:27.990168 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 21:24:27.990425 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:24:27.991659 25722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:28.117502 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 25599
I20260926 21:24:28.139099 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 25238
I20260926 21:24:28.141117 25300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:24:28.253499 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 25238
2026-09-26T21:24:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4922 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:24:28 dist-test-slave-4d5t krb5kdc[25779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:24:28 dist-test-slave-4d5t krb5kdc[25779](info): set up 2 sockets
Sep 26 21:24:28 dist-test-slave-4d5t krb5kdc[25779](info): commencing operation
krb5kdc: starting...
W20260926 21:24:29.421850 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:24:29 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:24:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:24:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/rangeradmin_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/rangeradmin_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/rangerlookup_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/rangerlookup_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
I20260926 21:24:29.794085 14904 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskh_JUDQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/postgres -l logfile start
2026-09-26 21:24:32.759 UTC [25809] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 21:24:32.760 UTC [25809] LOG: listening on IPv4 address "127.14.142.20", port 33075
2026-09-26 21:24:32.764 UTC [25809] LOG: listening on Unix socket "/tmp/.s.PGSQL.33075"
2026-09-26 21:24:32.787 UTC [25812] LOG: database system was shut down at 2026-09-26 21:24:30 UTC
2026-09-26 21:24:32.791 UTC [25809] LOG: database system is ready to accept connections
I20260926 21:24:33.821854 14904 mini_postgres.cc:96] Postgres bound to 33075
2026-09-26 21:24:33.841 UTC [25820] FATAL: database "slave" does not exist
127.14.142.20:33075 - accepting connections
I20260926 21:24:33.842358 14904 mini_ranger.cc:161] Starting Ranger...
I20260926 21:24:33.878279 14904 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 21:24:34.509770 14904 mini_ranger.cc:87] Created ranger Postgres database
I20260926 21:24:34.509949 14904 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-admin
2026-09-26 21:24:34,878 [I] DB FLAVOR :POSTGRES
2026-09-26 21:24:34,879 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 21:24:34,879 [I] Checking connection..
2026-09-26 21:24:34,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 21:24:35,183 [I] Checking connection passed.
2026-09-26 21:24:35,183 [I] --------- Verifying version history table ---------
2026-09-26 21:24:35,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 21:24:35,467 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 21:24:35,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 21:24:35,762 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 21:24:35,763 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 21:24:35,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 21:24:36.050 UTC [25884] WARNING: there is no transaction in progress
2026-09-26 21:24:36,057 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 21:24:36,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 21:24:36,325 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 21:24:36,326 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 21:24:36,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 21:24:36,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 21:24:36,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4d5t.c.gcp-upstream.internal','N') ;"
2026-09-26 21:24:37,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 21:24:37,457 [I] Table x_portal_user does not exist in database ranger
2026-09-26 21:24:37,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 21:24:37,723 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 21:24:37,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 21:24:37,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 21:24:38,278 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 21:24:38,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 21:24:39.384 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.404 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.424 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.443 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.464 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.488 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.559 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.567 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.583 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.596 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.608 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.625 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.637 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.645 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.654 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:39.662 UTC [25992] WARNING: there is no transaction in progress
2026-09-26 21:24:40,508 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 21:24:40,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 21:24:40,804 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 21:24:40,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 21:24:41,084 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 21:24:41,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 21:24:41,356 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 21:24:41,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 21:24:41,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 21:24:41,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4d5t.c.gcp-upstream.internal';"
2026-09-26 21:24:42,152 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 21:24:42,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 21:24:42,424 [I] DB_PATCHES have already been applied
I20260926 21:24:42.431427 14904 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-admin:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskh_JUDQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 21:24:42.431622 14904 mini_ranger.cc:193] Using host: 127.14.142.20
I20260926 21:24:42.452736 14904 mini_ranger.cc:227] Ranger admin URL: http://127.14.142.20:38987
Sep 26, 2026 9:24:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 9:24:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 9:24:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 9:24:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 9:24:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 9:24:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 9:24:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 21:24:43.620303 14904 mini_ranger.cc:160] Time spent starting Ranger: real 9.778s user 0.001s sys 0.050s
Sep 26, 2026 9:25:04 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 9:25:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 9:25:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 9:25:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 9:25:05 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 21:25:06.338217 14904 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/rangerkms_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/rangerkms_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.142.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.142.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 21:25:06 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457906, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 21:25:06.542485 14904 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 21:25:06.566594 14904 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 21:25:07.517764 14904 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 21:25:07.517922 14904 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms
I20260926 21:25:07.518092 14904 mini_ranger_kms.cc:227] Using postgres at 127.14.142.20:33075
2026-09-26 21:25:08,016 [I] DB FLAVOR :POSTGRES
2026-09-26 21:25:08,016 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 21:25:08,016 [I] Checking connection
2026-09-26 21:25:08,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 21:25:08,284 [I] connection success
2026-09-26 21:25:08,284 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 21:25:08,284 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 21:25:08,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 21:25:08,556 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 21:25:08,556 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 21:25:08,556 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 21:25:08,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:33075/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 21:25:08,846 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 21:25:09.452172 14904 mini_ranger_kms.cc:310] Created kms service
I20260926 21:25:09.671113 14904 mini_ranger_kms.cc:326] Created kudu user
I20260926 21:25:09.723258 14904 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 21:25:09 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457906, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:25:10.149925 14904 mini_ranger_kms.cc:384] Added ranger policy
I20260926 21:25:10.150257 14904 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms:/tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 21:25:10.150327 14904 mini_ranger_kms.cc:240] Using host: 127.14.142.20
I20260926 21:25:10.175205 14904 mini_ranger_kms.cc:276] Ranger KMS PID: 26173
I20260926 21:25:10.175442 14904 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.142.20:46633
Sep 26, 2026 9:25:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 9:25:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 9:25:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 9:25:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 9:25:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 9:25:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 21:25:12.578141 14904 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.036s user 0.138s sys 0.385s
Sep 26 21:25:12 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790457912, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.142.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:25:12.578375 14904 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 21:25:12.578563 14904 mini_ranger_kms.cc:401] 127.14.142.20:46633/kms/v1/keys
Sep 26 21:25:13 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457912, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.142.20@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
Sep 26 21:25:13 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457912, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.142.20@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
Sep 26 21:25:15 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:25:15.873586 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41659
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41659
--ranger_config_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:46633/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:25:16.236096 26223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:25:16.236426 26223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:25:16.236516 26223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:25:16.244112 26223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:25:16.244254 26223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:25:16.244300 26223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:25:16.244426 26223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:25:16.244498 26223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:25:16.255194 26223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.142.20:46633/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41659
--ranger_config_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:25:16.256716 26223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:25:16.258158 26223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:25:16.270570 26228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:16.270627 26229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:16.270613 26231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:25:16.271677 26223 server_base.cc:1034] running on GCE node
I20260926 21:25:16.272922 26223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:25:16.274660 26223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:25:16.275956 26223 hybrid_clock.cc:648] HybridClock initialized: now 1790457916275931 us; error 68 us; skew 500 ppm
I20260926 21:25:16.276360 26223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:25:16 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457916, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:25:16.285429 26223 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:25:16.287241 26223 webserver.cc:460] Webserver started at http://127.14.142.62:42941/ using document root <none> and password file <none>
I20260926 21:25:16.287964 26223 fs_manager.cc:359] Metadata directory not provided
I20260926 21:25:16.288108 26223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:25:16.288419 26223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:25:16 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457916, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:25:16.359236 26223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "fa51c6a7f19c494891067e7603e87b78"
format_stamp: "Formatted at 2026-09-26 21:25:16 on dist-test-slave-4d5t"
server_key: "601e26be58db4502a8cd1bbc78fb4e81"
server_key_iv: "67f24bdeeb7aa30a3d9acf852c7f4f1f"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:16.360633 26223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "fa51c6a7f19c494891067e7603e87b78"
format_stamp: "Formatted at 2026-09-26 21:25:16 on dist-test-slave-4d5t"
server_key: "601e26be58db4502a8cd1bbc78fb4e81"
server_key_iv: "67f24bdeeb7aa30a3d9acf852c7f4f1f"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:16.368952 26223 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260926 21:25:16.390771 26238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:16.392928 26223 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:25:16.393162 26223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "fa51c6a7f19c494891067e7603e87b78"
format_stamp: "Formatted at 2026-09-26 21:25:16 on dist-test-slave-4d5t"
server_key: "601e26be58db4502a8cd1bbc78fb4e81"
server_key_iv: "67f24bdeeb7aa30a3d9acf852c7f4f1f"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:16.393453 26223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:25:16.423270 26223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:25:16.429042 26223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:25:16.429420 26223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:25:16.451917 26223 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:41659
I20260926 21:25:16.452011 26289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:41659 every 8 connection(s)
I20260926 21:25:16.453459 26223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:25:16.459504 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:25:16.461898 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 26223
I20260926 21:25:16.462179 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:25:16.462905 14904 external_mini_cluster.cc:1428] Setting key 4a340c9472f16f2882e7319652d164ab
I20260926 21:25:16.475046 26290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78: Bootstrap starting.
Sep 26 21:25:16 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:25:16.479208 26290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78: Neither blocks nor log segments found. Creating new log.
I20260926 21:25:16.480579 26290 log.cc:824] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78: Log is configured to *not* fsync() on all Append() calls
I20260926 21:25:16.486099 26290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78: No bootstrap required, opened a new log
I20260926 21:25:16.489375 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:16.465906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.14.142.62:41659)
0926 21:25:16.466931 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:16.466944 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:16.466990 (+ 46us) server_negotiation.cc:407] Connection header received
0926 21:25:16.468953 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:16.468979 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:16.469421 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:16.469867 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:16.471354 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:16.473044 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:16.474496 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:16.475365 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:16.475574 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:16.480248 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:16.480258 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:16.480275 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:16.480322 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:16.484525 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:16.485427 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:16.485439 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:16.485454 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:16.485621 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:16.486047 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:16.486054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:16.486057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:16.486750 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:16.487095 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:16.487630 (+ 535us) server_negotiation.cc:299] Negotiation successful
0926 21:25:16.488130 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":477,"threads_started":1}
I20260926 21:25:16.497398 26290 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa51c6a7f19c494891067e7603e87b78" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41659 } }
I20260926 21:25:16.497838 26290 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:25:16.497951 26290 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa51c6a7f19c494891067e7603e87b78, State: Initialized, Role: FOLLOWER
I20260926 21:25:16.498548 26290 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa51c6a7f19c494891067e7603e87b78" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41659 } }
I20260926 21:25:16.498793 26290 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:25:16.498940 26290 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:25:16.499104 26290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:25:16.501567 26290 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa51c6a7f19c494891067e7603e87b78" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41659 } }
I20260926 21:25:16.502148 26290 leader_election.cc:302] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa51c6a7f19c494891067e7603e87b78; no voters:
I20260926 21:25:16.503252 26290 leader_election.cc:288] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:25:16.503408 26295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:25:16.504256 26295 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [term 1 LEADER]: Becoming Leader. State: Replica: fa51c6a7f19c494891067e7603e87b78, State: Running, Role: LEADER
I20260926 21:25:16.504917 26295 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa51c6a7f19c494891067e7603e87b78" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41659 } }
I20260926 21:25:16.505577 26290 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:25:16.508064 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa51c6a7f19c494891067e7603e87b78. Latest consensus state: current_term: 1 leader_uuid: "fa51c6a7f19c494891067e7603e87b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa51c6a7f19c494891067e7603e87b78" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41659 } } }
I20260926 21:25:16.508603 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [sys.catalog]: This master's current role is: LEADER
I20260926 21:25:16.508222 26296 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa51c6a7f19c494891067e7603e87b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa51c6a7f19c494891067e7603e87b78" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41659 } } }
I20260926 21:25:16.509227 26296 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78 [sys.catalog]: This master's current role is: LEADER
I20260926 21:25:16.514009 26304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:25:16.519726 26304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:25:16.522603 26290 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 21:25:16.542542 26304 catalog_manager.cc:1347] Generated new cluster ID: 764965a257e0478099d6a69b32074323
I20260926 21:25:16.542719 26304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:25:16.553798 26304 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:25:16.555889 26304 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:25:16.567559 26304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78: Generated new TSK 0
I20260926 21:25:16.568668 26304 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:25:17.490183 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:46633/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41659
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:25:17 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790457917, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 21:25:17.987494 26335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:25:17.987960 26335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:25:17.988188 26335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:25:18.000748 26335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:25:18.000917 26335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:25:18.001364 26335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:25:18.014488 26335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.142.20:46633/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:25:18.016278 26335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:25:18.018092 26335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:25:18.033255 26346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:18.033880 26348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:18.035027 26345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:25:18.035928 26335 server_base.cc:1034] running on GCE node
I20260926 21:25:18.037091 26335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:25:18.038666 26335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:25:18.040004 26335 hybrid_clock.cc:648] HybridClock initialized: now 1790457918039915 us; error 87 us; skew 500 ppm
I20260926 21:25:18.040397 26335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:25:18 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457918, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:25:18.044932 26335 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:25:18.047653 26335 webserver.cc:460] Webserver started at http://127.14.142.1:39987/ using document root <none> and password file <none>
I20260926 21:25:18.048666 26335 fs_manager.cc:359] Metadata directory not provided
I20260926 21:25:18.048823 26335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:25:18.049242 26335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:25:18 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457918, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:25:18.074949 26335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "37f49e38758c4165b3503a8ef8d855dc"
format_stamp: "Formatted at 2026-09-26 21:25:18 on dist-test-slave-4d5t"
server_key: "c5a5048b4c4f1f4a96d0c1833db96d5f"
server_key_iv: "6a5e070b51418a4cb1f97ffc29411ba9"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:18.076236 26335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "37f49e38758c4165b3503a8ef8d855dc"
format_stamp: "Formatted at 2026-09-26 21:25:18 on dist-test-slave-4d5t"
server_key: "c5a5048b4c4f1f4a96d0c1833db96d5f"
server_key_iv: "6a5e070b51418a4cb1f97ffc29411ba9"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:18.084360 26335 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.003s
I20260926 21:25:18.104511 26355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:18.107263 26335 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260926 21:25:18.107522 26335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "37f49e38758c4165b3503a8ef8d855dc"
format_stamp: "Formatted at 2026-09-26 21:25:18 on dist-test-slave-4d5t"
server_key: "c5a5048b4c4f1f4a96d0c1833db96d5f"
server_key_iv: "6a5e070b51418a4cb1f97ffc29411ba9"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:18.107858 26335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:25:18.150730 26335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:25:18.156500 26335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:25:18.156947 26335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:25:18.164392 26335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:25:18.167344 26335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:25:18.167536 26335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:18.167760 26335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:25:18.167903 26335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:25:18 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457917, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:25:18.325016 26335 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:34245
I20260926 21:25:18.325065 26467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:34245 every 8 connection(s)
I20260926 21:25:18.327011 26335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:25:18.335906 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 26335
I20260926 21:25:18.336193 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:25:18.337558 14904 external_mini_cluster.cc:1428] Setting key ef8f2ea166653560bcfaeba917934775
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Sep 26 21:25:18 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457918, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:25:18.439888 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:18.362822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:53451 (local address 127.14.142.62:41659)
0926 21:25:18.389413 (+ 26591us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:18.389423 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:18.392982 (+ 3559us) server_negotiation.cc:407] Connection header received
0926 21:25:18.408789 (+ 15807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:18.408800 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:18.408959 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:18.409073 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:18.416775 (+ 7702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:18.418085 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:18.421187 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:18.422013 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:18.422137 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:18.430540 (+ 8403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:18.430552 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:18.430562 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:18.430621 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:18.435178 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:18.436407 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:18.436416 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:18.436423 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:18.436624 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:18.437255 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:18.437261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:18.437264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:18.437684 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:18.437846 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:18.439065 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0926 21:25:18.439363 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26375,"thread_start_us":26604,"threads_started":1}
I20260926 21:25:18.442678 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:18.361983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41659 (local address 127.14.142.1:53451)
0926 21:25:18.390561 (+ 28578us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:18.390643 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:18.405910 (+ 15267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:18.409926 (+ 4016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:18.409946 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:18.410905 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:25:18.412523 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:18.412605 (+ 82us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:18.418387 (+ 5782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:18.418397 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:18.420930 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:18.420965 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:18.422648 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:18.422655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:18.423008 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:18.423920 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:25:18.423959 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:25:18.430153 (+ 6194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:25:18.435441 (+ 5288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:25:18.435456 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:25:18.435482 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:25:18.436218 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:25:18.436816 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:25:18.436825 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:25:18.436832 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:25:18.437112 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:25:18.437963 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:25:18.437976 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:25:18.438535 (+ 559us) client_negotiation.cc:769] Sending connection context
0926 21:25:18.440296 (+ 1761us) client_negotiation.cc:240] Negotiation successful
0926 21:25:18.441034 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28084,"thread_start_us":39511,"threads_started":1}
I20260926 21:25:18.445206 26468 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41659
I20260926 21:25:18.445564 26468 heartbeater.cc:464] Registering TS with master...
I20260926 21:25:18.446308 26468 heartbeater.cc:511] Master 127.14.142.62:41659 requested a full tablet report, sending...
I20260926 21:25:18.448786 26255 ts_manager.cc:194] Registered new tserver with Master: 37f49e38758c4165b3503a8ef8d855dc (127.14.142.1:34245)
I20260926 21:25:18.451031 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:53451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
W20260926 21:25:18.472204 14904 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.14.142.2: real 0.131s user 0.002s sys 0.016s
I20260926 21:25:18.473078 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:46633/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41659
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:25:18 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457917, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
W20260926 21:25:18.981745 26474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:25:18.982177 26474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:25:18.982373 26474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:25:18.994019 26474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:25:18.994175 26474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:25:18.994357 26474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:25:19.009933 26474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.142.20:46633/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:25:19.011663 26474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:25:19.013381 26474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:25:19.028491 26491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:19.028796 26488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:19.028466 26489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:25:19.030659 26474 server_base.cc:1034] running on GCE node
I20260926 21:25:19.031311 26474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:25:19.032315 26474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:25:19.033566 26474 hybrid_clock.cc:648] HybridClock initialized: now 1790457919033515 us; error 42 us; skew 500 ppm
I20260926 21:25:19.033915 26474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:25:19 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:25:19.037853 26474 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:25:19.039523 26474 webserver.cc:460] Webserver started at http://127.14.142.2:39389/ using document root <none> and password file <none>
I20260926 21:25:19.040246 26474 fs_manager.cc:359] Metadata directory not provided
I20260926 21:25:19.040376 26474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:25:19.040750 26474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:25:19 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:25:19.058874 26474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "860f0d0a4551468e9de1ea0ebcf6ef73"
format_stamp: "Formatted at 2026-09-26 21:25:19 on dist-test-slave-4d5t"
server_key: "377b158e89c1c480d738f85fd8b5752f"
server_key_iv: "15712e3856b9a127290ee9e1951c4694"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:19.060477 26474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "860f0d0a4551468e9de1ea0ebcf6ef73"
format_stamp: "Formatted at 2026-09-26 21:25:19 on dist-test-slave-4d5t"
server_key: "377b158e89c1c480d738f85fd8b5752f"
server_key_iv: "15712e3856b9a127290ee9e1951c4694"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:19.066493 26474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260926 21:25:19.085366 26498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:19.087015 26474 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.003s
I20260926 21:25:19.087232 26474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "860f0d0a4551468e9de1ea0ebcf6ef73"
format_stamp: "Formatted at 2026-09-26 21:25:19 on dist-test-slave-4d5t"
server_key: "377b158e89c1c480d738f85fd8b5752f"
server_key_iv: "15712e3856b9a127290ee9e1951c4694"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:19.087873 26474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:25:19.109807 26474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:25:19.114094 26474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:25:19.114501 26474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:25:19.115550 26474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:25:19.117026 26474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:25:19.117132 26474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:19.117350 26474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:25:19.117432 26474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:19.158030 26474 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:32843
I20260926 21:25:19.158011 26610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:32843 every 8 connection(s)
I20260926 21:25:19.160099 26474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:25:19.166653 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 26474
I20260926 21:25:19.166853 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:25:19.167299 14904 external_mini_cluster.cc:1428] Setting key 1d513fa4a3ebeeaafd12d275f29f5f05
Sep 26 21:25:19 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:25:19.183320 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.163485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:56383 (local address 127.14.142.62:41659)
0926 21:25:19.164004 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:19.164009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:19.165080 (+ 1071us) server_negotiation.cc:407] Connection header received
0926 21:25:19.167214 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:19.167220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:19.167330 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:19.167464 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:19.169430 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.170350 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.171957 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.172378 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.172516 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.176738 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:19.176744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:19.176748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:19.176787 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:19.179983 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:19.180929 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:19.180934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:19.180936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:19.181062 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:19.181518 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:19.181528 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:19.181530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:19.181813 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:19.181970 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:19.182703 (+ 733us) server_negotiation.cc:299] Negotiation successful
0926 21:25:19.182940 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":298,"threads_started":1}
I20260926 21:25:19.184463 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.164053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41659 (local address 127.14.142.2:56383)
0926 21:25:19.164911 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:19.164956 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:19.166913 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:19.167715 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:19.167724 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:19.168288 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:25:19.169255 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.169276 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.170498 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.170504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.171806 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.171818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.172656 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.172665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.173325 (+ 660us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.174080 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:25:19.174104 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:25:19.176415 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:25:19.180177 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:25:19.180187 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:25:19.180218 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:25:19.180774 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:25:19.181175 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:25:19.181181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:25:19.181184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:25:19.181394 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:25:19.181941 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:25:19.181949 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:25:19.182310 (+ 361us) client_negotiation.cc:769] Sending connection context
0926 21:25:19.182589 (+ 279us) client_negotiation.cc:240] Negotiation successful
0926 21:25:19.183144 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":373,"threads_started":1}
I20260926 21:25:19.186262 26611 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41659
I20260926 21:25:19.186597 26611 heartbeater.cc:464] Registering TS with master...
I20260926 21:25:19.187340 26611 heartbeater.cc:511] Master 127.14.142.62:41659 requested a full tablet report, sending...
I20260926 21:25:19.189109 26255 ts_manager.cc:194] Registered new tserver with Master: 860f0d0a4551468e9de1ea0ebcf6ef73 (127.14.142.2:32843)
I20260926 21:25:19.190452 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:56383
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:25:19.224764 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:46633/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41659
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:34211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:25:19.454706 26468 heartbeater.cc:503] Master 127.14.142.62:41659 was elected leader, sending a full tablet report...
W20260926 21:25:19.614670 26617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:25:19.615017 26617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:25:19.615185 26617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:25:19.625470 26617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:25:19.625640 26617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:25:19.625830 26617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:25:19.637094 26617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:34211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.142.20:46633/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:25:19.638437 26617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:25:19.639806 26617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:25:19.654400 26622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:19.654461 26623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:25:19.654426 26625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:25:19.655180 26617 server_base.cc:1034] running on GCE node
I20260926 21:25:19.655768 26617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:25:19.656762 26617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:25:19.657986 26617 hybrid_clock.cc:648] HybridClock initialized: now 1790457919657943 us; error 30 us; skew 500 ppm
I20260926 21:25:19.658340 26617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:25:19 dist-test-slave-4d5t krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:25:19.662536 26617 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:25:19.664199 26617 webserver.cc:460] Webserver started at http://127.14.142.3:46445/ using document root <none> and password file <none>
I20260926 21:25:19.665020 26617 fs_manager.cc:359] Metadata directory not provided
I20260926 21:25:19.665151 26617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:25:19.665477 26617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:25:19 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:25:19.680693 26617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "d8c015de79734dcea4dd852d23ee47ba"
format_stamp: "Formatted at 2026-09-26 21:25:19 on dist-test-slave-4d5t"
server_key: "b7cab25b3f911e088d62849849e2bcc9"
server_key_iv: "44cb3dceab0889ed572bea00efa1b004"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:19.681779 26617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "d8c015de79734dcea4dd852d23ee47ba"
format_stamp: "Formatted at 2026-09-26 21:25:19 on dist-test-slave-4d5t"
server_key: "b7cab25b3f911e088d62849849e2bcc9"
server_key_iv: "44cb3dceab0889ed572bea00efa1b004"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:19.687495 26617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:25:19.704793 26632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:19.706468 26617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 21:25:19.706653 26617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "d8c015de79734dcea4dd852d23ee47ba"
format_stamp: "Formatted at 2026-09-26 21:25:19 on dist-test-slave-4d5t"
server_key: "b7cab25b3f911e088d62849849e2bcc9"
server_key_iv: "44cb3dceab0889ed572bea00efa1b004"
server_key_version: "kuduclusterkey@0"
I20260926 21:25:19.706859 26617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:25:19.722033 26617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:25:19.726068 26617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:25:19.726383 26617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:25:19.727447 26617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:25:19.728953 26617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:25:19.729027 26617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:19.729185 26617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:25:19.729260 26617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:25:19.770625 26617 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:40797
I20260926 21:25:19.771126 26744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:40797 every 8 connection(s)
I20260926 21:25:19.772308 26617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:25:19.782696 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 26617
I20260926 21:25:19.782915 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:25:19.783460 14904 external_mini_cluster.cc:1428] Setting key 9de0987115bb3422a748aeb263c896e3
Sep 26 21:25:19 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457919, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:25:19.794507 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.775105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:55413 (local address 127.14.142.62:41659)
0926 21:25:19.775784 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:19.775790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:19.776618 (+ 828us) server_negotiation.cc:407] Connection header received
0926 21:25:19.778674 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:19.778679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:19.778783 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:19.778898 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:19.780350 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.781315 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.782923 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.783610 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.783759 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.787731 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:19.787737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:19.787740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:19.787777 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:19.790758 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:19.791897 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:19.791902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:19.791904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:19.792104 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:19.792807 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:19.792811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:19.792813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:19.793108 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:19.793233 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:19.794058 (+ 825us) server_negotiation.cc:299] Negotiation successful
0926 21:25:19.794245 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":360,"threads_started":1}
I20260926 21:25:19.795573 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.775639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41659 (local address 127.14.142.3:55413)
0926 21:25:19.776412 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:19.776456 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:19.778316 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:19.779142 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:19.779150 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:19.779593 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:25:19.780157 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.780174 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.781463 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.781467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.782711 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.782723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.783871 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.783878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.784163 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.785140 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:25:19.785168 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:25:19.787491 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:25:19.791075 (+ 3584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:25:19.791083 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:25:19.791100 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:25:19.791774 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:25:19.792374 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:25:19.792377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:25:19.792380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:25:19.792699 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:25:19.793306 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:25:19.793313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:25:19.793657 (+ 344us) client_negotiation.cc:769] Sending connection context
0926 21:25:19.793923 (+ 266us) client_negotiation.cc:240] Negotiation successful
0926 21:25:19.794365 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":330,"threads_started":1}
I20260926 21:25:19.797149 26745 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41659
I20260926 21:25:19.797502 26745 heartbeater.cc:464] Registering TS with master...
I20260926 21:25:19.798437 26745 heartbeater.cc:511] Master 127.14.142.62:41659 requested a full tablet report, sending...
I20260926 21:25:19.800046 26255 ts_manager.cc:194] Registered new tserver with Master: d8c015de79734dcea4dd852d23ee47ba (127.14.142.3:40797)
I20260926 21:25:19.800959 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:55413
I20260926 21:25:19.812227 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:25:19.833348 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.817755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.14.142.62:41659)
0926 21:25:19.817946 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:19.817951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:19.818600 (+ 649us) server_negotiation.cc:407] Connection header received
0926 21:25:19.818834 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:19.818839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:19.818941 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:19.819052 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:19.821244 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.822031 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.823520 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.824079 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.824226 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.827082 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:19.827087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:19.827091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:19.827127 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:19.830348 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:19.831152 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:19.831160 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:19.831164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:19.831307 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:19.831758 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:19.831767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:19.831771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:19.832089 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:19.832377 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:19.832737 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 21:25:19.832995 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:25:19.837960 26255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:25:19.845948 26255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:25:19.874836 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.860570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:32843 (local address 127.0.0.1:41118)
0926 21:25:19.862003 (+ 1433us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:19.862059 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:19.862495 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:19.864679 (+ 2184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:19.864691 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:19.864743 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:25:19.865435 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.865454 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.868219 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.868227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.871616 (+ 3389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.871637 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.873784 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.873791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.874009 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.874065 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 21:25:19.874199 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 21:25:19.874427 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":839,"threads_started":1}
I20260926 21:25:19.875248 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.860466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.14.142.2:32843)
0926 21:25:19.863723 (+ 3257us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:19.863735 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:19.863771 (+ 36us) server_negotiation.cc:407] Connection header received
0926 21:25:19.863925 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:19.863936 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:19.864250 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:19.864494 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:25:19.865882 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.867986 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.871859 (+ 3873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.873538 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.873727 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.873940 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:19.874778 (+ 838us) server_negotiation.cc:299] Negotiation successful
0926 21:25:19.874974 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3108,"thread_start_us":3012,"threads_started":1}
I20260926 21:25:19.879643 26546 tablet_service.cc:1467] Processing CreateTablet for tablet 7b93c7a8b63146f4978e6f40ec21bda6 (DEFAULT_TABLE table=test-table [id=d02fc95735424613b453a8297d32dadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:25:19.881338 26546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b93c7a8b63146f4978e6f40ec21bda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:25:19.881618 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.862397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:40797 (local address 127.0.0.1:36372)
0926 21:25:19.873747 (+ 11350us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:19.873777 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:19.873916 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:19.875376 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:19.875382 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:19.875408 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:25:19.875756 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.875767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.877426 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.877432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.878867 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.878885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.880769 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.880774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.880950 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.880994 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 21:25:19.881099 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 21:25:19.881249 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11216,"thread_start_us":11572,"threads_started":1}
I20260926 21:25:19.882000 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.861259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.14.142.1:34245)
0926 21:25:19.873308 (+ 12049us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:19.873317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:19.873363 (+ 46us) server_negotiation.cc:407] Connection header received
0926 21:25:19.873481 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:19.873494 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:19.873799 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:19.874655 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:25:19.874879 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.876357 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.878961 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.880641 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.880778 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.881019 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:19.881490 (+ 471us) server_negotiation.cc:299] Negotiation successful
0926 21:25:19.881654 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11846,"thread_start_us":11032,"threads_started":1}
I20260926 21:25:19.882190 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.860588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:34245 (local address 127.0.0.1:49212)
0926 21:25:19.863108 (+ 2520us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:19.863128 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:19.863252 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:19.874026 (+ 10774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:19.874030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:19.874053 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:25:19.874471 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.874484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.876496 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.876504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.878730 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:19.878746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.881120 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.881125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:19.881287 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.881316 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 21:25:19.881414 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 21:25:19.881539 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2370,"thread_start_us":2485,"threads_started":1}
I20260926 21:25:19.883476 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:19.862776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.14.142.3:40797)
0926 21:25:19.873469 (+ 10693us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:19.873479 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:19.874052 (+ 573us) server_negotiation.cc:407] Connection header received
0926 21:25:19.874171 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:19.874178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:19.874406 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:19.874598 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:25:19.875958 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.877258 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.879073 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:19.880528 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:19.880711 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:19.880955 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:19.881736 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 21:25:19.882547 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10525,"thread_start_us":10373,"threads_started":1}
I20260926 21:25:19.886417 26403 tablet_service.cc:1467] Processing CreateTablet for tablet 7b93c7a8b63146f4978e6f40ec21bda6 (DEFAULT_TABLE table=test-table [id=d02fc95735424613b453a8297d32dadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:25:19.886902 26680 tablet_service.cc:1467] Processing CreateTablet for tablet 7b93c7a8b63146f4978e6f40ec21bda6 (DEFAULT_TABLE table=test-table [id=d02fc95735424613b453a8297d32dadb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:25:19.888055 26403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b93c7a8b63146f4978e6f40ec21bda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:25:19.888187 26680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b93c7a8b63146f4978e6f40ec21bda6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:25:19.897362 26764 tablet_bootstrap.cc:492] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Bootstrap starting.
I20260926 21:25:19.900849 26764 tablet_bootstrap.cc:654] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Neither blocks nor log segments found. Creating new log.
I20260926 21:25:19.902629 26764 log.cc:824] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Log is configured to *not* fsync() on all Append() calls
I20260926 21:25:19.904861 26766 tablet_bootstrap.cc:492] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Bootstrap starting.
I20260926 21:25:19.905649 26765 tablet_bootstrap.cc:492] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Bootstrap starting.
I20260926 21:25:19.906132 26764 tablet_bootstrap.cc:492] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: No bootstrap required, opened a new log
I20260926 21:25:19.906415 26764 ts_tablet_manager.cc:1397] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260926 21:25:19.908674 26765 tablet_bootstrap.cc:654] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Neither blocks nor log segments found. Creating new log.
I20260926 21:25:19.908668 26766 tablet_bootstrap.cc:654] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Neither blocks nor log segments found. Creating new log.
I20260926 21:25:19.910012 26766 log.cc:824] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Log is configured to *not* fsync() on all Append() calls
I20260926 21:25:19.910274 26765 log.cc:824] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Log is configured to *not* fsync() on all Append() calls
I20260926 21:25:19.913400 26766 tablet_bootstrap.cc:492] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: No bootstrap required, opened a new log
I20260926 21:25:19.913717 26766 ts_tablet_manager.cc:1397] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260926 21:25:19.914059 26765 tablet_bootstrap.cc:492] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: No bootstrap required, opened a new log
I20260926 21:25:19.914425 26765 ts_tablet_manager.cc:1397] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260926 21:25:19.917703 26764 raft_consensus.cc:348] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:19.918177 26764 raft_consensus.cc:374] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:25:19.918318 26764 raft_consensus.cc:729] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860f0d0a4551468e9de1ea0ebcf6ef73, State: Initialized, Role: FOLLOWER
I20260926 21:25:19.918988 26764 consensus_queue.cc:260] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:19.921361 26611 heartbeater.cc:503] Master 127.14.142.62:41659 was elected leader, sending a full tablet report...
I20260926 21:25:19.922123 26764 ts_tablet_manager.cc:1428] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Time spent starting tablet: real 0.016s user 0.006s sys 0.011s
I20260926 21:25:19.923260 26766 raft_consensus.cc:348] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:19.923756 26766 raft_consensus.cc:374] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:25:19.923867 26766 raft_consensus.cc:729] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c015de79734dcea4dd852d23ee47ba, State: Initialized, Role: FOLLOWER
I20260926 21:25:19.924593 26766 consensus_queue.cc:260] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:19.925319 26765 raft_consensus.cc:348] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:19.925810 26765 raft_consensus.cc:374] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:25:19.925889 26745 heartbeater.cc:503] Master 127.14.142.62:41659 was elected leader, sending a full tablet report...
I20260926 21:25:19.925989 26765 raft_consensus.cc:729] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37f49e38758c4165b3503a8ef8d855dc, State: Initialized, Role: FOLLOWER
I20260926 21:25:19.926760 26766 ts_tablet_manager.cc:1428] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260926 21:25:19.926700 26765 consensus_queue.cc:260] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:19.929113 26765 ts_tablet_manager.cc:1428] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260926 21:25:20.008744 26770 raft_consensus.cc:482] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:25:20.009145 26770 raft_consensus.cc:504] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:20.010995 26770 leader_election.cc:288] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37f49e38758c4165b3503a8ef8d855dc (127.14.142.1:34245), d8c015de79734dcea4dd852d23ee47ba (127.14.142.3:40797)
I20260926 21:25:20.020059 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.011478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43603 (local address 127.14.142.3:40797)
0926 21:25:20.011706 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:20.011712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:20.013129 (+ 1417us) server_negotiation.cc:407] Connection header received
0926 21:25:20.013354 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:20.013365 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:20.013444 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:20.013554 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:25:20.014343 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.015052 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.017831 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.018829 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.019023 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.019095 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:20.019672 (+ 577us) server_negotiation.cc:299] Negotiation successful
0926 21:25:20.019817 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:25:20.020079 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.011398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:34245 (local address 127.14.142.2:41365)
0926 21:25:20.012838 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:20.012857 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:20.012999 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:20.013367 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:20.013372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:20.013398 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:25:20.013816 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:20.013826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.015349 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.015353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:20.017482 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:20.017494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.019050 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.019056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:20.019345 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.019372 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 21:25:20.019476 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 21:25:20.019611 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":3,"thread_start_us":712,"threads_started":1}
I20260926 21:25:20.020089 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.011457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:41365 (local address 127.14.142.1:34245)
0926 21:25:20.011706 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:20.011713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:20.012964 (+ 1251us) server_negotiation.cc:407] Connection header received
0926 21:25:20.013122 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:20.013127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:20.013215 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:20.013314 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:25:20.013965 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.015207 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.017661 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.018796 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.018968 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.019072 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:20.019599 (+ 527us) server_negotiation.cc:299] Negotiation successful
0926 21:25:20.019826 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 21:25:20.020262 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.011604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:40797 (local address 127.14.142.2:43603)
0926 21:25:20.012997 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0926 21:25:20.013018 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:25:20.013156 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:25:20.013595 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:25:20.013599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:25:20.013619 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:25:20.014083 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:20.014097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.015198 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.015204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:20.017670 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:25:20.017684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.019108 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.019115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:25:20.019396 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.019422 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 21:25:20.019611 (+ 189us) client_negotiation.cc:240] Negotiation successful
0926 21:25:20.019833 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"mutex_wait_us":111,"thread_start_us":641,"threads_started":1}
I20260926 21:25:20.021221 26423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b93c7a8b63146f4978e6f40ec21bda6" candidate_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f49e38758c4165b3503a8ef8d855dc" is_pre_election: true
I20260926 21:25:20.021500 26700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b93c7a8b63146f4978e6f40ec21bda6" candidate_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c015de79734dcea4dd852d23ee47ba" is_pre_election: true
I20260926 21:25:20.021883 26423 raft_consensus.cc:2454] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 860f0d0a4551468e9de1ea0ebcf6ef73 in term 0.
I20260926 21:25:20.021962 26700 raft_consensus.cc:2454] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 860f0d0a4551468e9de1ea0ebcf6ef73 in term 0.
I20260926 21:25:20.022666 26500 leader_election.cc:302] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37f49e38758c4165b3503a8ef8d855dc, 860f0d0a4551468e9de1ea0ebcf6ef73; no voters:
I20260926 21:25:20.023085 26770 raft_consensus.cc:2781] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:25:20.023240 26770 raft_consensus.cc:482] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:25:20.023351 26770 raft_consensus.cc:3037] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 0 FOLLOWER]: Advancing to term 1
W20260926 21:25:20.024021 26746 tablet.cc:2369] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:25:20.025986 26770 raft_consensus.cc:504] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:20.026646 26770 leader_election.cc:288] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [CANDIDATE]: Term 1 election: Requested vote from peers 37f49e38758c4165b3503a8ef8d855dc (127.14.142.1:34245), d8c015de79734dcea4dd852d23ee47ba (127.14.142.3:40797)
I20260926 21:25:20.027144 26423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b93c7a8b63146f4978e6f40ec21bda6" candidate_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f49e38758c4165b3503a8ef8d855dc"
I20260926 21:25:20.027211 26700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b93c7a8b63146f4978e6f40ec21bda6" candidate_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c015de79734dcea4dd852d23ee47ba"
I20260926 21:25:20.027386 26700 raft_consensus.cc:3037] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:25:20.027391 26423 raft_consensus.cc:3037] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:25:20.030748 26700 raft_consensus.cc:2454] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 860f0d0a4551468e9de1ea0ebcf6ef73 in term 1.
I20260926 21:25:20.031332 26423 raft_consensus.cc:2454] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 860f0d0a4551468e9de1ea0ebcf6ef73 in term 1.
I20260926 21:25:20.031818 26502 leader_election.cc:302] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 37f49e38758c4165b3503a8ef8d855dc, 860f0d0a4551468e9de1ea0ebcf6ef73, d8c015de79734dcea4dd852d23ee47ba; no voters:
I20260926 21:25:20.032199 26770 raft_consensus.cc:2781] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:25:20.032667 26770 raft_consensus.cc:686] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 1 LEADER]: Becoming Leader. State: Replica: 860f0d0a4551468e9de1ea0ebcf6ef73, State: Running, Role: LEADER
I20260926 21:25:20.033242 26770 consensus_queue.cc:237] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } }
I20260926 21:25:20.038327 26253 catalog_manager.cc:5495] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 860f0d0a4551468e9de1ea0ebcf6ef73 (127.14.142.2). New cstate: current_term: 1 leader_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "860f0d0a4551468e9de1ea0ebcf6ef73" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 32843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 } health_report { overall_health: UNKNOWN } } }
I20260926 21:25:20.068847 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.062911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.14.142.2:32843)
0926 21:25:20.063104 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:20.063108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:20.063126 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:25:20.063285 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:20.063288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:20.063360 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:20.063488 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:25:20.064134 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.064999 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.066618 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.067029 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.067125 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.067727 (+ 602us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:25:20.068239 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:25:20.068352 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:20.068485 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 21:25:20.068574 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 21:25:20.077509 26700 raft_consensus.cc:1261] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 1 FOLLOWER]: Refusing update from remote peer 860f0d0a4551468e9de1ea0ebcf6ef73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:25:20.077519 26423 raft_consensus.cc:1261] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 1 FOLLOWER]: Refusing update from remote peer 860f0d0a4551468e9de1ea0ebcf6ef73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:25:20.078487 26770 consensus_queue.cc:1035] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37f49e38758c4165b3503a8ef8d855dc" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 34245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:25:20.078824 26775 consensus_queue.cc:1035] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8c015de79734dcea4dd852d23ee47ba" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 40797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:25:20.096077 26779 mvcc.cc:205] Tried to move back new op lower bound from 7333715640627961856 to 7333715640460115968. Current Snapshot: MvccSnapshot[applied={T|T < 7333715640627961856}]
I20260926 21:25:20.108832 26253 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:25:20.109243 26253 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:25:20.113849 26253 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa51c6a7f19c494891067e7603e87b78: Sending DeleteTablet for 3 replicas of tablet 7b93c7a8b63146f4978e6f40ec21bda6
I20260926 21:25:20.115314 26546 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b93c7a8b63146f4978e6f40ec21bda6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:25:20 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:41118
I20260926 21:25:20.115309 26403 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b93c7a8b63146f4978e6f40ec21bda6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:25:20 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:49212
I20260926 21:25:20.116392 26787 tablet_replica.cc:330] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: stopping tablet replica
I20260926 21:25:20.116863 26787 raft_consensus.cc:2229] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:25:20.116890 26680 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b93c7a8b63146f4978e6f40ec21bda6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:25:20 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:36372
I20260926 21:25:20.117408 26787 raft_consensus.cc:2258] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:25:20.118209 26788 tablet_replica.cc:330] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: stopping tablet replica
I20260926 21:25:20.118924 26788 raft_consensus.cc:2229] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:25:20.119622 26788 raft_consensus.cc:2258] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:25:20.120800 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 26335
I20260926 21:25:20.121824 26787 ts_tablet_manager.cc:1905] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:25:20.124878 26788 ts_tablet_manager.cc:1905] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:25:20.127738 26789 tablet_replica.cc:330] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: stopping tablet replica
I20260926 21:25:20.128476 26789 raft_consensus.cc:2229] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:25:20.129217 26789 raft_consensus.cc:2258] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:25:20.131223 26787 ts_tablet_manager.cc:1918] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:25:20.131404 26787 log.cc:1196] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/7b93c7a8b63146f4978e6f40ec21bda6
I20260926 21:25:20.131887 26789 ts_tablet_manager.cc:1905] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:25:20.131943 26787 ts_tablet_manager.cc:1939] T 7b93c7a8b63146f4978e6f40ec21bda6 P 37f49e38758c4165b3503a8ef8d855dc: Deleting consensus metadata
I20260926 21:25:20.133836 26240 catalog_manager.cc:4841] TS 37f49e38758c4165b3503a8ef8d855dc (127.14.142.1:34245): tablet 7b93c7a8b63146f4978e6f40ec21bda6 (table test-table [id=d02fc95735424613b453a8297d32dadb]) successfully deleted
I20260926 21:25:20.139329 26789 ts_tablet_manager.cc:1918] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:25:20.139643 26789 log.cc:1196] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/7b93c7a8b63146f4978e6f40ec21bda6
I20260926 21:25:20.140225 26789 ts_tablet_manager.cc:1939] T 7b93c7a8b63146f4978e6f40ec21bda6 P d8c015de79734dcea4dd852d23ee47ba: Deleting consensus metadata
I20260926 21:25:20.142256 26788 ts_tablet_manager.cc:1918] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:25:20.142284 26242 catalog_manager.cc:4841] TS d8c015de79734dcea4dd852d23ee47ba (127.14.142.3:40797): tablet 7b93c7a8b63146f4978e6f40ec21bda6 (table test-table [id=d02fc95735424613b453a8297d32dadb]) successfully deleted
I20260926 21:25:20.142621 26788 log.cc:1196] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/7b93c7a8b63146f4978e6f40ec21bda6
I20260926 21:25:20.143203 26788 ts_tablet_manager.cc:1939] T 7b93c7a8b63146f4978e6f40ec21bda6 P 860f0d0a4551468e9de1ea0ebcf6ef73: Deleting consensus metadata
Sep 26 21:25:20 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:25:20.145048 26242 catalog_manager.cc:4841] TS 860f0d0a4551468e9de1ea0ebcf6ef73 (127.14.142.2:32843): tablet 7b93c7a8b63146f4978e6f40ec21bda6 (table test-table [id=d02fc95735424613b453a8297d32dadb]) successfully deleted
I20260926 21:25:20.152202 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.123937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.14.142.1:34245)
0926 21:25:20.124162 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:20.124172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:20.134677 (+ 10505us) server_negotiation.cc:407] Connection header received
0926 21:25:20.134967 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:20.134973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:20.135049 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:20.135121 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:20.137093 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.138248 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.140272 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.140953 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.141056 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.145958 (+ 4902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:20.145973 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:20.145988 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:20.146022 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:20.149137 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:20.150027 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:20.150034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:20.150036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:20.150172 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:20.150587 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:20.150592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:20.150594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:20.150965 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:20.151274 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:20.151575 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 21:25:20.151848 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:25:20.153026 26463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:25:20.273537 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 26335
I20260926 21:25:20.297266 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 26474
Sep 26 21:25:20 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:25:20.317147 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.298118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.14.142.2:32843)
0926 21:25:20.298324 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:20.298331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:20.298353 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:25:20.298512 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:20.298518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:20.298644 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:20.298768 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:20.300671 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.302061 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.303490 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.304236 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.304393 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.309995 (+ 5602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:20.310018 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:20.310042 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:20.310100 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:20.313982 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:20.314839 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:20.314856 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:20.314860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:20.315052 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:20.315434 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:20.315440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:20.315443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:20.315821 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:20.316221 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:20.316503 (+ 282us) server_negotiation.cc:299] Negotiation successful
0926 21:25:20.316757 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 21:25:20.318231 26606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:25:20.437835 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 26474
I20260926 21:25:20.458384 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 26617
Sep 26 21:25:20 dist-test-slave-4d5t krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:25:20.476089 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:25:20.459461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.14.142.3:40797)
0926 21:25:20.459665 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 21:25:20.459671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:25:20.459694 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:25:20.459831 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:25:20.459837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:25:20.459922 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:25:20.460031 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:25:20.461456 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.462540 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.464472 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:25:20.465243 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:25:20.465372 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:25:20.469802 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:25:20.469818 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:25:20.469836 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:25:20.469877 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:25:20.473067 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:20.473894 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:20.473909 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:20.473911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:20.474059 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:25:20.474434 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:25:20.474438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:25:20.474440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:25:20.474827 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:25:20.475036 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 21:25:20.475563 (+ 527us) server_negotiation.cc:299] Negotiation successful
0926 21:25:20.475728 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 21:25:20.477038 26740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:25:20.568192 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 26617
I20260926 21:25:20.582620 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 26223
I20260926 21:25:20.583822 26285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:25:20.702562 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 26223
I20260926 21:25:20.731920 14904 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 21:25:21.079059 14904 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 21:25:21.079418 14904 mini_ranger.cc:66] Stopping Ranger...
I20260926 21:25:21.426021 14904 mini_ranger.cc:68] Stopped Ranger
2026-09-26 21:25:21.426 UTC [25809] LOG: received smart shutdown request
2026-09-26 21:25:21.430 UTC [25809] LOG: background worker "logical replication launcher" (PID 25818) exited with exit code 1
2026-09-26 21:25:21.430 UTC [25813] LOG: shutting down
2026-09-26 21:25:21.631 UTC [25809] LOG: database system is shut down
2026-09-26T21:25:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53521 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:25:21 dist-test-slave-4d5t krb5kdc[26803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:25:21 dist-test-slave-4d5t krb5kdc[26803](info): set up 2 sockets
Sep 26 21:25:21 dist-test-slave-4d5t krb5kdc[26803](info): commencing operation
krb5kdc: starting...
W20260926 21:25:22.929225 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:25:23 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:25:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:25:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/rangeradmin_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/rangeradmin_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/rangerlookup_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/rangerlookup_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
I20260926 21:25:23.288300 14904 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskh_JUDQ/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/postgres -l logfile start
2026-09-26 21:25:26.057 UTC [26833] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 21:25:26.057 UTC [26833] LOG: listening on IPv4 address "127.14.142.20", port 46521
2026-09-26 21:25:26.061 UTC [26833] LOG: listening on Unix socket "/tmp/.s.PGSQL.46521"
2026-09-26 21:25:26.087 UTC [26836] LOG: database system was shut down at 2026-09-26 21:25:24 UTC
2026-09-26 21:25:26.091 UTC [26833] LOG: database system is ready to accept connections
I20260926 21:25:27.145438 14904 mini_postgres.cc:96] Postgres bound to 46521
2026-09-26 21:25:27.165 UTC [26844] FATAL: database "slave" does not exist
127.14.142.20:46521 - accepting connections
I20260926 21:25:27.166299 14904 mini_ranger.cc:161] Starting Ranger...
I20260926 21:25:27.199291 14904 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 21:25:27.763258 14904 mini_ranger.cc:87] Created ranger Postgres database
I20260926 21:25:27.763440 14904 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-admin
2026-09-26 21:25:28,061 [I] DB FLAVOR :POSTGRES
2026-09-26 21:25:28,061 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 21:25:28,062 [I] Checking connection..
2026-09-26 21:25:28,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 21:25:28,311 [I] Checking connection passed.
2026-09-26 21:25:28,311 [I] --------- Verifying version history table ---------
2026-09-26 21:25:28,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 21:25:28,556 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 21:25:28,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 21:25:28,803 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 21:25:28,803 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 21:25:28,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 21:25:29.073 UTC [26908] WARNING: there is no transaction in progress
2026-09-26 21:25:29,080 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 21:25:29,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 21:25:29,346 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 21:25:29,346 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 21:25:29,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 21:25:29,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 21:25:29,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-4d5t.c.gcp-upstream.internal','N') ;"
2026-09-26 21:25:30,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 21:25:30,433 [I] Table x_portal_user does not exist in database ranger
2026-09-26 21:25:30,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 21:25:30,733 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 21:25:30,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 21:25:30,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 21:25:31,262 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 21:25:31,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 21:25:32.358 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.376 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.396 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.415 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.433 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.453 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.517 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.525 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.541 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.553 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.564 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.579 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.591 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.599 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.608 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:32.616 UTC [27016] WARNING: there is no transaction in progress
2026-09-26 21:25:33,381 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 21:25:33,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 21:25:33,638 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 21:25:33,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 21:25:33,921 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 21:25:33,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 21:25:34,173 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 21:25:34,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 21:25:34,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 21:25:34,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-4d5t.c.gcp-upstream.internal';"
2026-09-26 21:25:34,925 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 21:25:34,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 21:25:35,198 [I] DB_PATCHES have already been applied
I20260926 21:25:35.205370 14904 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-admin:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskh_JUDQ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 21:25:35.205498 14904 mini_ranger.cc:193] Using host: 127.14.142.20
I20260926 21:25:35.221848 14904 mini_ranger.cc:227] Ranger admin URL: http://127.14.142.20:55287
Sep 26, 2026 9:25:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 9:25:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 9:25:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 9:25:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 9:25:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 9:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 9:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 21:25:36.431205 14904 mini_ranger.cc:160] Time spent starting Ranger: real 9.265s user 0.002s sys 0.047s
Sep 26, 2026 9:25:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 9:25:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 26, 2026 9:25:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 9:25:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 9:25:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 21:25:59.152597 14904 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.14.142.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.14.142.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/rangerkms_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.14.142.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/rangerkms_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.14.142.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.14.142.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.14.142.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/HTTP_127.14.142.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 21:25:59 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457959, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 21:25:59.363670 14904 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 21:25:59.386394 14904 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 21:26:00.418885 14904 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 21:26:00.419121 14904 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms
I20260926 21:26:00.419222 14904 mini_ranger_kms.cc:227] Using postgres at 127.14.142.20:46521
2026-09-26 21:26:00,807 [I] DB FLAVOR :POSTGRES
2026-09-26 21:26:00,808 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 21:26:00,808 [I] Checking connection
2026-09-26 21:26:00,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 21:26:01,077 [I] connection success
2026-09-26 21:26:01,077 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 21:26:01,077 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 21:26:01,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 21:26:01,325 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 21:26:01,326 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 21:26:01,326 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 21:26:01,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.14.142.20:46521/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 21:26:01,627 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 21:26:02.173843 14904 mini_ranger_kms.cc:310] Created kms service
I20260926 21:26:02.392405 14904 mini_ranger_kms.cc:326] Created kudu user
I20260926 21:26:02.439231 14904 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 21:26:02 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457959, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:02.920406 14904 mini_ranger_kms.cc:384] Added ranger policy
I20260926 21:26:02.920783 14904 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms:/tmp/dist-test-taskh_JUDQ/build/asan/bin/postgresql.jar:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskh_JUDQ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 21:26:02.920850 14904 mini_ranger_kms.cc:240] Using host: 127.14.142.20
I20260926 21:26:02.940857 14904 mini_ranger_kms.cc:276] Ranger KMS PID: 27197
I20260926 21:26:02.941011 14904 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.14.142.20:40033
Sep 26, 2026 9:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 9:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 9:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 9:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 9:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 9:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 21:26:05.293156 14904 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.929s user 0.087s sys 0.315s
I20260926 21:26:05.293422 14904 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 21:26:05.293607 14904 mini_ranger_kms.cc:401] 127.14.142.20:40033/kms/v1/keys
Sep 26 21:26:05 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790457965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.142.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 21:26:05 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.142.20@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
Sep 26 21:26:06 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.14.142.20@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
Sep 26 21:26:08 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:26:08.841549 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41095
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41095
--ranger_config_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:09.205348 27247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:09.205756 27247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:09.205931 27247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:09.213760 27247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:09.214210 27247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:09.214388 27247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:09.214501 27247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:09.214586 27247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:09.226166 27247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
--ranger_kms_url=127.14.142.20:40033/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41095
--ranger_config_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:09.227602 27247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:09.229053 27247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:09.243286 27252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:09.243360 27253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:09.243332 27255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:09.244705 27247 server_base.cc:1034] running on GCE node
I20260926 21:26:09.245817 27247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:09.248342 27247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:09.249631 27247 hybrid_clock.cc:648] HybridClock initialized: now 1790457969249576 us; error 42 us; skew 500 ppm
I20260926 21:26:09.250101 27247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:09 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457969, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:09.255590 27247 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:09.258266 27247 webserver.cc:460] Webserver started at http://127.14.142.62:46845/ using document root <none> and password file <none>
I20260926 21:26:09.259363 27247 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:09.259508 27247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:09.259905 27247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:26:09 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457969, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:09.329862 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "c1ee0e6fef254260b9c3b4acb476b5eb"
format_stamp: "Formatted at 2026-09-26 21:26:09 on dist-test-slave-4d5t"
server_key: "66cc5a60294e6ba26824d95c0018d2e6"
server_key_iv: "654f9f6c6bb75b635d1dcff9f39333c9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:09.331049 27247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "c1ee0e6fef254260b9c3b4acb476b5eb"
format_stamp: "Formatted at 2026-09-26 21:26:09 on dist-test-slave-4d5t"
server_key: "66cc5a60294e6ba26824d95c0018d2e6"
server_key_iv: "654f9f6c6bb75b635d1dcff9f39333c9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:09.338449 27247 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260926 21:26:09.360072 27262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:09.362092 27247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:26:09.362303 27247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "c1ee0e6fef254260b9c3b4acb476b5eb"
format_stamp: "Formatted at 2026-09-26 21:26:09 on dist-test-slave-4d5t"
server_key: "66cc5a60294e6ba26824d95c0018d2e6"
server_key_iv: "654f9f6c6bb75b635d1dcff9f39333c9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:09.362593 27247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:09.422493 27247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:09.427908 27247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:09.428313 27247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:09.450095 27247 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:41095
I20260926 21:26:09.450086 27313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:41095 every 8 connection(s)
I20260926 21:26:09.452049 27247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:09.453037 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 27247
I20260926 21:26:09.453281 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:26:09.454135 14904 external_mini_cluster.cc:1428] Setting key 4ce6704a03644188420ef3762a32f8cc
I20260926 21:26:09.464285 27314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:09.479228 27314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Bootstrap starting.
Sep 26 21:26:09 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:09.483274 27314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:09.484542 27314 log.cc:824] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:09.487999 27314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: No bootstrap required, opened a new log
I20260926 21:26:09.490886 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:09.456506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.14.142.62:41095)
0926 21:26:09.464636 (+ 8130us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:09.464657 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:09.465343 (+ 686us) server_negotiation.cc:407] Connection header received
0926 21:26:09.467902 (+ 2559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:09.467933 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:09.468515 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:09.469012 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:09.470882 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:09.473017 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:09.474904 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:09.476337 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:09.476613 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:09.481790 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:09.481802 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:09.481822 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:09.481879 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:09.485926 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:09.486823 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:09.486832 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:09.486842 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:09.487035 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:09.487494 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:09.487500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:09.487503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:09.488204 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:09.488496 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:09.489032 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 21:26:09.489438 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7733,"thread_start_us":7735,"threads_started":1}
I20260926 21:26:09.496898 27314 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:09.497284 27314 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:09.497427 27314 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ee0e6fef254260b9c3b4acb476b5eb, State: Initialized, Role: FOLLOWER
I20260926 21:26:09.498239 27314 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:09.498534 27314 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:09.498699 27314 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:09.498971 27314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:09.501818 27314 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:09.502362 27314 leader_election.cc:302] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ee0e6fef254260b9c3b4acb476b5eb; no voters:
I20260926 21:26:09.503039 27314 leader_election.cc:288] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:26:09.503147 27319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:09.504458 27319 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 LEADER]: Becoming Leader. State: Replica: c1ee0e6fef254260b9c3b4acb476b5eb, State: Running, Role: LEADER
I20260926 21:26:09.505182 27319 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:09.505573 27314 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:09.508665 27320 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } } }
I20260926 21:26:09.509042 27320 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:09.508517 27321 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ee0e6fef254260b9c3b4acb476b5eb. Latest consensus state: current_term: 1 leader_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } } }
I20260926 21:26:09.509377 27321 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:09.525527 27326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:09.531685 27326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:09.533064 27314 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 21:26:09.567986 27326 catalog_manager.cc:1347] Generated new cluster ID: 197a3fa1d58c45039a506d3cef004309
I20260926 21:26:09.568320 27326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:09.583105 27326 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:26:09.585415 27326 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:09.595937 27326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Generated new TSK 0
I20260926 21:26:09.597644 27326 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:26:10.493165 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41095
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:26:10 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790457970, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 21:26:10.929032 27359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:10.929379 27359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:10.929546 27359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:10.937299 27359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:10.937466 27359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:10.937638 27359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:26:10.948499 27359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.14.142.20:40033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:10.949954 27359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:10.951283 27359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:10.977152 27369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:10.977145 27370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:10.977758 27372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:10.979019 27359 server_base.cc:1034] running on GCE node
I20260926 21:26:10.979759 27359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:10.981215 27359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:10.982529 27359 hybrid_clock.cc:648] HybridClock initialized: now 1790457970982464 us; error 62 us; skew 500 ppm
I20260926 21:26:10.982903 27359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:10 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457970, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:10.987169 27359 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:26:10.989866 27359 webserver.cc:460] Webserver started at http://127.14.142.1:33789/ using document root <none> and password file <none>
I20260926 21:26:10.990597 27359 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:10.990752 27359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:10.991081 27359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:26:10 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457970, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:11.017915 27359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "e4be445cf919484e85f6bec392e31364"
format_stamp: "Formatted at 2026-09-26 21:26:11 on dist-test-slave-4d5t"
server_key: "120a2492c2d321115114564510944468"
server_key_iv: "77abc1c3886a675d492841b0f5b3bc31"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:11.019099 27359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "e4be445cf919484e85f6bec392e31364"
format_stamp: "Formatted at 2026-09-26 21:26:11 on dist-test-slave-4d5t"
server_key: "120a2492c2d321115114564510944468"
server_key_iv: "77abc1c3886a675d492841b0f5b3bc31"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:11.042924 27359 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.007s sys 0.015s
I20260926 21:26:11.085372 27379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:11.092916 27359 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.009s sys 0.007s
I20260926 21:26:11.093251 27359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "e4be445cf919484e85f6bec392e31364"
format_stamp: "Formatted at 2026-09-26 21:26:11 on dist-test-slave-4d5t"
server_key: "120a2492c2d321115114564510944468"
server_key_iv: "77abc1c3886a675d492841b0f5b3bc31"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:11.093649 27359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:11.113677 27359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:11.120086 27359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:11.120488 27359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:11.123190 27359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:11.125439 27359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:11.125550 27359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:11.125689 27359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:11.125777 27359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:26:11 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457970, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:11.530326 27359 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:39977
I20260926 21:26:11.530369 27491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:39977 every 8 connection(s)
I20260926 21:26:11.531982 27359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:26:11.537025 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 27359
I20260926 21:26:11.537237 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:26:11.538215 14904 external_mini_cluster.cc:1428] Setting key 38200eb8e8f90b3b7b3e7c6f3abe6e42
Sep 26 21:26:11 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457970, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:11.565874 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:11.534951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:50127 (local address 127.14.142.62:41095)
0926 21:26:11.535805 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:11.535815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:11.539514 (+ 3699us) server_negotiation.cc:407] Connection header received
0926 21:26:11.546681 (+ 7167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:11.546687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:11.546778 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:11.546854 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:11.549141 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:11.550224 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:11.552495 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:11.553185 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:11.553321 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:11.558502 (+ 5181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:11.558509 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:11.558521 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:11.558569 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:11.562041 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:11.563009 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:11.563014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:11.563017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:11.563170 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:11.563685 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:11.563689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:11.563692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:11.564037 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:11.564177 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:11.565267 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0926 21:26:11.565509 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":464,"threads_started":1}
I20260926 21:26:11.568605 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:11.535519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41095 (local address 127.14.142.1:50127)
0926 21:26:11.539312 (+ 3793us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:11.539376 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:11.546282 (+ 6906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:11.547446 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:11.547458 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:11.548065 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:11.548916 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:11.548939 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:11.550420 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:11.550425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:11.552300 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:11.552313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:11.553652 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:11.553659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:11.553955 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:11.554771 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:11.554802 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:11.557894 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:11.562210 (+ 4316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:11.562220 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:11.562240 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:11.562840 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:11.563302 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:11.563307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:11.563310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:11.563556 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:11.564266 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:11.564274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:11.564738 (+ 464us) client_negotiation.cc:769] Sending connection context
0926 21:26:11.566424 (+ 1686us) client_negotiation.cc:240] Negotiation successful
0926 21:26:11.567021 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":412,"threads_started":1}
I20260926 21:26:11.571048 27492 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41095
I20260926 21:26:11.571357 27492 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:11.572189 27492 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:26:11.575059 27279 ts_manager.cc:194] Registered new tserver with Master: e4be445cf919484e85f6bec392e31364 (127.14.142.1:39977)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:26:11.576889 27279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:50127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:26:11.609230 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41095
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:26:11 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457970, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
W20260926 21:26:12.117588 27498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:12.118119 27498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:12.118297 27498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:12.130579 27498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:12.130887 27498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:12.131167 27498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:26:12.148214 27498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.14.142.20:40033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:12.150125 27498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:12.151918 27498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:12.169924 27505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:12.171015 27506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:12.188737 27508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:12.189007 27498 server_base.cc:1034] running on GCE node
I20260926 21:26:12.189904 27498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:12.199110 27498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:12.200651 27498 hybrid_clock.cc:648] HybridClock initialized: now 1790457972200520 us; error 124 us; skew 500 ppm
I20260926 21:26:12.201151 27498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457972, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:12.205490 27498 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:26:12.210136 27498 webserver.cc:460] Webserver started at http://127.14.142.2:43519/ using document root <none> and password file <none>
I20260926 21:26:12.211217 27498 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:12.211364 27498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:12.211702 27498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457972, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:12.231714 27498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "155160e9a7a241b78dceb8f3c9f8d53e"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "6e03d6fffe10018964bfa8cf72331407"
server_key_iv: "874b62fdcf56e30a629ff44453c60340"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:12.233129 27498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "155160e9a7a241b78dceb8f3c9f8d53e"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "6e03d6fffe10018964bfa8cf72331407"
server_key_iv: "874b62fdcf56e30a629ff44453c60340"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:12.241497 27498 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260926 21:26:12.260650 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:12.262668 27498 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260926 21:26:12.262892 27498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "155160e9a7a241b78dceb8f3c9f8d53e"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "6e03d6fffe10018964bfa8cf72331407"
server_key_iv: "874b62fdcf56e30a629ff44453c60340"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:12.263176 27498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:12.291913 27498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:12.297431 27498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:12.297783 27498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:12.299041 27498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:12.300985 27498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:12.301083 27498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:12.301254 27498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:12.301316 27498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:12.348026 27498 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33401
I20260926 21:26:12.348047 27634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33401 every 8 connection(s)
I20260926 21:26:12.349429 27498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:26:12.356709 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 27498
I20260926 21:26:12.356894 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:26:12.357412 14904 external_mini_cluster.cc:1428] Setting key 4429fcd5d43a2ba34e9582e558193e2d
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457972, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:12.375626 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:12.352300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:50467 (local address 127.14.142.62:41095)
0926 21:26:12.352839 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:12.352844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:12.353888 (+ 1044us) server_negotiation.cc:407] Connection header received
0926 21:26:12.355747 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:12.355754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:12.355849 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:12.355971 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:12.357634 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.358600 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.361369 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.362020 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.362160 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:12.367339 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:12.367347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:12.367352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:12.367396 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:12.371603 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:12.372760 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:12.372766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:12.372769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:12.372930 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:12.373486 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:12.373491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:12.373494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:12.373865 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:12.374024 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:12.374982 (+ 958us) server_negotiation.cc:299] Negotiation successful
0926 21:26:12.375236 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":316,"threads_started":1}
I20260926 21:26:12.377018 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:12.352943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41095 (local address 127.14.142.2:50467)
0926 21:26:12.353717 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:12.353754 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:12.355448 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:12.356211 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:12.356219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:12.356721 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:12.357439 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:12.357457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.358782 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.358788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:12.360859 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:12.360897 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.362523 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.362532 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:12.362887 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:12.363711 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:12.363760 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:12.366954 (+ 3194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:12.371804 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:12.371817 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:12.371839 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:12.372596 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:12.373055 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:12.373063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:12.373070 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:12.373339 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:12.374021 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:12.374032 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:12.374499 (+ 467us) client_negotiation.cc:769] Sending connection context
0926 21:26:12.374824 (+ 325us) client_negotiation.cc:240] Negotiation successful
0926 21:26:12.375432 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":319,"threads_started":1}
I20260926 21:26:12.378839 27635 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41095
I20260926 21:26:12.379174 27635 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:12.380064 27635 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
I20260926 21:26:12.381882 27279 ts_manager.cc:194] Registered new tserver with Master: 155160e9a7a241b78dceb8f3c9f8d53e (127.14.142.2:33401)
I20260926 21:26:12.383028 27279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:50467
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:26:12.419889 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41095
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:12.580924 27492 heartbeater.cc:503] Master 127.14.142.62:41095 was elected leader, sending a full tablet report...
W20260926 21:26:12.772898 27641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:12.773186 27641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:12.773280 27641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:12.780820 27641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:12.780949 27641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:12.781119 27641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:26:12.791131 27641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.14.142.20:40033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:12.792680 27641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:12.794023 27641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:12.809502 27647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:12.809427 27649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:12.810020 27646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:12.811069 27641 server_base.cc:1034] running on GCE node
I20260926 21:26:12.811956 27641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:12.813199 27641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:12.814442 27641 hybrid_clock.cc:648] HybridClock initialized: now 1790457972814346 us; error 73 us; skew 500 ppm
I20260926 21:26:12.814854 27641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457972, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:12.818290 27641 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:26:12.820140 27641 webserver.cc:460] Webserver started at http://127.14.142.3:42223/ using document root <none> and password file <none>
I20260926 21:26:12.820920 27641 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:12.821033 27641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:12.821285 27641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457972, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:12.841562 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "01fd217eb49545e09201749506160c8c"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "2388dd1ec2fe3485c8ccca8488913293"
server_key_iv: "08192e769b06321c58813bddc65ed8b9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:12.842775 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "01fd217eb49545e09201749506160c8c"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "2388dd1ec2fe3485c8ccca8488913293"
server_key_iv: "08192e769b06321c58813bddc65ed8b9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:12.850479 27641 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260926 21:26:12.868278 27656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:12.870404 27641 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 21:26:12.870605 27641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "01fd217eb49545e09201749506160c8c"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "2388dd1ec2fe3485c8ccca8488913293"
server_key_iv: "08192e769b06321c58813bddc65ed8b9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:12.870873 27641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:12.897511 27641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:12.902622 27641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:12.903023 27641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:12.904259 27641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:12.906256 27641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:12.906435 27641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:12.906591 27641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:12.906673 27641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:12.945292 27641 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:38571
I20260926 21:26:12.945322 27768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:38571 every 8 connection(s)
I20260926 21:26:12.947050 27641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:26:12.956990 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 27641
I20260926 21:26:12.957384 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:26:12.957996 14904 external_mini_cluster.cc:1428] Setting key 09a2f734e8d41eafe2e6e0aea2bb18b9
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457972, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:12.970046 27771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:12.950196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:41409 (local address 127.14.142.62:41095)
0926 21:26:12.950710 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:12.950714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:12.951839 (+ 1125us) server_negotiation.cc:407] Connection header received
0926 21:26:12.953746 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:12.953752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:12.953952 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:12.954096 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:12.955858 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.956915 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.959090 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.959696 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.959838 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:12.963420 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:12.963425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:12.963429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:12.963468 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:12.966693 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:12.967492 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:12.967499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:12.967506 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:12.967681 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:12.968169 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:12.968180 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:12.968183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:12.968458 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:12.968663 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:12.969416 (+ 753us) server_negotiation.cc:299] Negotiation successful
0926 21:26:12.969679 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":306,"threads_started":1}
I20260926 21:26:12.971036 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:12.950832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41095 (local address 127.14.142.3:41409)
0926 21:26:12.951661 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:12.951705 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:12.953456 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:12.954356 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:12.954365 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:12.954827 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:12.955437 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:12.955454 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.957131 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.957136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:12.958931 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:12.958946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.959957 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.959965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:12.960211 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:12.960962 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:12.960991 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:12.963158 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:12.966880 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:12.966888 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:12.966904 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:12.967363 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:12.967825 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:12.967829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:12.967831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:12.968038 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:12.968635 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:12.968643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:12.969044 (+ 401us) client_negotiation.cc:769] Sending connection context
0926 21:26:12.969344 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 21:26:12.969884 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":344,"threads_started":1}
I20260926 21:26:12.972584 27769 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41095
I20260926 21:26:12.972976 27769 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:12.973976 27769 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
I20260926 21:26:12.975565 27279 ts_manager.cc:194] Registered new tserver with Master: 01fd217eb49545e09201749506160c8c (127.14.142.3:38571)
I20260926 21:26:12.976416 27279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:41409
I20260926 21:26:12.987339 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:26:12.989125 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 27359
Sep 26 21:26:12 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:26:13.008440 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:12.990627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.14.142.1:39977)
0926 21:26:12.991315 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:12.991322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:12.991348 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:26:12.991407 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:12.991413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:12.991839 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:12.992032 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:12.993620 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.994682 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.996505 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:12.997169 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:12.997283 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:13.002253 (+ 4970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:13.002266 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:13.002281 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:13.002320 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:13.005368 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:13.006280 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:13.006289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:13.006291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:13.006451 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:13.007021 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:13.007025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:13.007027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:13.007391 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:13.007745 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:13.007936 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 21:26:13.008148 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":376,"threads_started":1}
I20260926 21:26:13.009877 27487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:13.103528 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 27359
I20260926 21:26:13.122583 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 27498
Sep 26 21:26:13 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:26:13.142035 27776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:13.124135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.14.142.2:33401)
0926 21:26:13.124883 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:13.124892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:13.124925 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:26:13.125000 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:13.125008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:13.125299 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:13.125466 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:13.126950 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:13.128145 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:13.129965 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:13.130709 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:13.130851 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:13.135558 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:13.135577 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:13.135594 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:13.135646 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:13.138725 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:13.139480 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:13.139488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:13.139490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:13.139663 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:13.140112 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:13.140116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:13.140118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:13.140532 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:13.140879 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:13.141489 (+ 610us) server_negotiation.cc:299] Negotiation successful
0926 21:26:13.141732 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":378,"threads_started":1}
I20260926 21:26:13.143416 27630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:13.229450 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 27498
I20260926 21:26:13.250638 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 27641
Sep 26 21:26:13 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:26:13.269109 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:13.251713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41576 (local address 127.14.142.3:38571)
0926 21:26:13.252378 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:13.252385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:13.252416 (+ 31us) server_negotiation.cc:407] Connection header received
0926 21:26:13.252472 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:13.252479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:13.252801 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:13.252976 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:13.254363 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:13.255389 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:13.256848 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:13.257424 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:13.257568 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:13.262694 (+ 5126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:13.262709 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:13.262725 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:13.262764 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:13.266062 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:13.266840 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:13.266851 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:13.266853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:13.266996 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:13.267388 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:13.267391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:13.267393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:13.267779 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:13.268030 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:13.268409 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 21:26:13.268702 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":356,"threads_started":1}
I20260926 21:26:13.270543 27764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:13.356034 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 27641
I20260926 21:26:13.373437 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 27247
I20260926 21:26:13.374796 27309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:13.485173 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 27247
I20260926 21:26:13.506966 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41095
--webserver_interface=127.14.142.62
--webserver_port=46845
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41095
--ranger_config_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:13.852679 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:13.853260 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:13.853477 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:13.861340 27781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:13.861476 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:13.861594 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:13.861672 27781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:13.861720 27781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:13.872714 27781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41095
--ranger_config_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=46845
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:13.874120 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:13.875500 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:13.889026 27786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:13.888989 27787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:13.889067 27789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:13.890388 27781 server_base.cc:1034] running on GCE node
I20260926 21:26:13.891426 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:13.893191 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:13.894441 27781 hybrid_clock.cc:648] HybridClock initialized: now 1790457973894393 us; error 40 us; skew 500 ppm
I20260926 21:26:13.894899 27781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:13 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457973, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:13.899055 27781 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:13.900725 27781 webserver.cc:460] Webserver started at http://127.14.142.62:46845/ using document root <none> and password file <none>
I20260926 21:26:13.901590 27781 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:13.901731 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 21:26:13 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457973, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:13.959932 27781 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:26:13.964313 27797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:13.966507 27781 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 21:26:13.966760 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "c1ee0e6fef254260b9c3b4acb476b5eb"
format_stamp: "Formatted at 2026-09-26 21:26:09 on dist-test-slave-4d5t"
server_key: "66cc5a60294e6ba26824d95c0018d2e6"
server_key_iv: "654f9f6c6bb75b635d1dcff9f39333c9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:13.968180 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:13.996914 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:14.001360 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:14.001626 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:14.023653 27781 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:41095
I20260926 21:26:14.023816 27848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:41095 every 8 connection(s)
I20260926 21:26:14.025636 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:14.032997 27849 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:26:14.034979 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 27781
I20260926 21:26:14.035902 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:39977
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:41095
--webserver_port=33789
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:14.036716 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Bootstrap starting.
I20260926 21:26:14.064221 27849 log.cc:824] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:14.076370 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:26:14.077096 27849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Bootstrap complete.
I20260926 21:26:14.090869 27849 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:14.092664 27849 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1ee0e6fef254260b9c3b4acb476b5eb, State: Initialized, Role: FOLLOWER
I20260926 21:26:14.093747 27849 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:14.094100 27849 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:14.094237 27849 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:14.094434 27849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 1 FOLLOWER]: Advancing to term 2
I20260926 21:26:14.100581 27849 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:14.101220 27849 leader_election.cc:302] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1ee0e6fef254260b9c3b4acb476b5eb; no voters:
I20260926 21:26:14.101871 27849 leader_election.cc:288] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 21:26:14.102057 27853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 2 FOLLOWER]: Leader election won for term 2
I20260926 21:26:14.103469 27853 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [term 2 LEADER]: Becoming Leader. State: Replica: c1ee0e6fef254260b9c3b4acb476b5eb, State: Running, Role: LEADER
I20260926 21:26:14.103986 27849 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:14.104130 27853 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } }
I20260926 21:26:14.106531 27855 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1ee0e6fef254260b9c3b4acb476b5eb. Latest consensus state: current_term: 2 leader_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } } }
I20260926 21:26:14.106906 27855 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:14.107375 27854 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1ee0e6fef254260b9c3b4acb476b5eb" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41095 } } }
I20260926 21:26:14.107728 27854 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:14.122154 27858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:14.128290 27858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:14.129808 27858 catalog_manager.cc:1259] Loaded cluster ID: 197a3fa1d58c45039a506d3cef004309
I20260926 21:26:14.129952 27858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:14.134127 27858 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:14.135772 27858 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Loaded TSK: 0
I20260926 21:26:14.137758 27858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:26:14.140676 27849 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260926 21:26:14.542423 27851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:14.542902 27851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:14.543118 27851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:14.555245 27851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:14.555469 27851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:14.555704 27851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:26:14.572582 27851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:39977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=33789
--tserver_master_addrs=127.14.142.62:41095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:14.574394 27851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:14.576371 27851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:14.605710 27887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:14.610352 27888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:14.616312 27851 server_base.cc:1034] running on GCE node
W20260926 21:26:14.617137 27890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:14.618116 27851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:14.619481 27851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:14.620780 27851 hybrid_clock.cc:648] HybridClock initialized: now 1790457974620671 us; error 97 us; skew 500 ppm
I20260926 21:26:14.621196 27851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:14 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457974, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:14.629944 27851 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:26:14.632261 27851 webserver.cc:460] Webserver started at http://127.14.142.1:33789/ using document root <none> and password file <none>
I20260926 21:26:14.633383 27851 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:14.633582 27851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 21:26:14 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457974, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:14.679299 27851 fs_manager.cc:709] Time spent opening directory manager: real 0.022s user 0.008s sys 0.012s
I20260926 21:26:14.684044 27898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:14.686249 27851 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260926 21:26:14.686498 27851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "e4be445cf919484e85f6bec392e31364"
format_stamp: "Formatted at 2026-09-26 21:26:11 on dist-test-slave-4d5t"
server_key: "120a2492c2d321115114564510944468"
server_key_iv: "77abc1c3886a675d492841b0f5b3bc31"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:14.688131 27851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:14.719904 27851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:14.725724 27851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:14.726100 27851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:14.727578 27851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:14.729612 27851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:14.729776 27851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 21:26:14.730086 27851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:14.730428 27851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:14.790247 27851 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:39977
I20260926 21:26:14.790279 28011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:39977 every 8 connection(s)
I20260926 21:26:14.792423 27851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:26:14.802865 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 27851
I20260926 21:26:14.803834 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:33401
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:41095
--webserver_port=43519
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:26:14 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457974, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:14.854883 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:14.799064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41095 (local address 127.14.142.1:60399)
0926 21:26:14.822166 (+ 23102us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:14.822252 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:14.825986 (+ 3734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:14.828946 (+ 2960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:14.828956 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:14.829708 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:14.830640 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:14.830670 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:14.833568 (+ 2898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:14.833575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:14.835993 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:14.836029 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:14.838368 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:14.838378 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:14.838729 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:14.839636 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:14.839674 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:14.842947 (+ 3273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:14.847896 (+ 4949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:14.847906 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:14.847928 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:14.848670 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:14.849334 (+ 664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:14.849339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:14.849343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:14.849630 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:14.850832 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:14.850841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:14.851271 (+ 430us) client_negotiation.cc:769] Sending connection context
0926 21:26:14.851576 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 21:26:14.852207 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22623,"thread_start_us":29392,"threads_started":1}
I20260926 21:26:14.854871 28015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:14.799055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:60399 (local address 127.14.142.62:41095)
0926 21:26:14.821254 (+ 22199us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:14.821303 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:14.822543 (+ 1240us) server_negotiation.cc:407] Connection header received
0926 21:26:14.826992 (+ 4449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:14.827035 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:14.827632 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:14.828029 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:14.830922 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:14.832986 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:14.836235 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:14.837597 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:14.837820 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:14.843370 (+ 5550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:14.843382 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:14.843406 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:14.843464 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:14.847638 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:14.848900 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:14.848911 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:14.848919 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:14.849169 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:14.849786 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:14.849791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:14.849794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:14.850558 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:14.850745 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:14.852916 (+ 2171us) server_negotiation.cc:299] Negotiation successful
0926 21:26:14.853424 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21574,"thread_start_us":22191,"threads_started":1}
I20260926 21:26:14.860750 28012 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41095
I20260926 21:26:14.861124 28012 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:14.862069 28012 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
W20260926 21:26:14.865893 28012 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:41095 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 21:26:14.866192 28012 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:14.866578 28012 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
I20260926 21:26:14.868386 28012 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:14.868786 28012 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
W20260926 21:26:14.871642 28012 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 21:26:15.250756 28016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:15.251091 28016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:15.251266 28016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:15.260695 28016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:15.260933 28016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:15.261152 28016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:26:15.273051 28016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:33401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=43519
--tserver_master_addrs=127.14.142.62:41095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:15.274829 28016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:15.277113 28016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:15.298547 28030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:15.302930 28031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:15.305825 28033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:15.306345 28016 server_base.cc:1034] running on GCE node
I20260926 21:26:15.307180 28016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:15.308496 28016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:15.309934 28016 hybrid_clock.cc:648] HybridClock initialized: now 1790457975309802 us; error 75 us; skew 500 ppm
I20260926 21:26:15.310563 28016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:15 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457975, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:15.314688 28016 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:26:15.317911 28016 webserver.cc:460] Webserver started at http://127.14.142.2:43519/ using document root <none> and password file <none>
I20260926 21:26:15.319052 28016 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:15.319252 28016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 21:26:15 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457975, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:15.346743 28016 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 21:26:15.352612 28041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:15.354427 28016 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260926 21:26:15.354664 28016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "155160e9a7a241b78dceb8f3c9f8d53e"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "6e03d6fffe10018964bfa8cf72331407"
server_key_iv: "874b62fdcf56e30a629ff44453c60340"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:15.356441 28016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:15.403861 28016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:15.411015 28016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:15.411471 28016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:15.421121 28016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:15.423431 28016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:15.423885 28016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 21:26:15.424329 28016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:15.424526 28016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 21:26:15 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790457975, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:15.492991 28016 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33401
I20260926 21:26:15.493058 28155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33401 every 8 connection(s)
I20260926 21:26:15.495129 28016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:26:15.504434 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28016
I20260926 21:26:15.505865 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.14.142.20:40033/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:38571
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:41095
--webserver_port=42223
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:26:15 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457975, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:15.576972 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:15.507951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:33053 (local address 127.14.142.62:41095)
0926 21:26:15.535450 (+ 27499us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:15.535459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:15.540506 (+ 5047us) server_negotiation.cc:407] Connection header received
0926 21:26:15.540621 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:15.540627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:15.540748 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:15.540858 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:15.546262 (+ 5404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:15.547576 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:15.560062 (+ 12486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:15.560950 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:15.561118 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:15.567416 (+ 6298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:15.567430 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:15.567438 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:15.567500 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:15.572366 (+ 4866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:15.573664 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:15.573671 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:15.573675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:15.573875 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:15.574522 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:15.574529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:15.574533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:15.574941 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:15.575103 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:15.576255 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0926 21:26:15.576522 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27313,"thread_start_us":27525,"threads_started":1}
I20260926 21:26:15.578650 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:15.507203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41095 (local address 127.14.142.2:33053)
0926 21:26:15.537093 (+ 29890us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:15.537172 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:15.539918 (+ 2746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:15.543488 (+ 3570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:15.543507 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:15.544637 (+ 1130us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:15.545914 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:15.545958 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:15.557031 (+ 11073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:15.557042 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:15.559698 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:15.559752 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:15.561597 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:15.561608 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:15.562028 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:15.562973 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:15.563028 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:15.566996 (+ 3968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:15.572696 (+ 5700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:15.572708 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:15.572733 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:15.573482 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:15.574022 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:15.574028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:15.574032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:15.574343 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:15.575209 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:15.575219 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:15.575745 (+ 526us) client_negotiation.cc:769] Sending connection context
0926 21:26:15.576099 (+ 354us) client_negotiation.cc:240] Negotiation successful
0926 21:26:15.576810 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29469,"thread_start_us":34461,"threads_started":1}
I20260926 21:26:15.580636 28156 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41095
I20260926 21:26:15.580991 28156 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:15.581957 28156 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
I20260926 21:26:15.584368 27814 ts_manager.cc:194] Registered new tserver with Master: 155160e9a7a241b78dceb8f3c9f8d53e (127.14.142.2:33401)
I20260926 21:26:15.586609 27814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:33053
I20260926 21:26:15.871793 28012 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:15.872337 28012 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
I20260926 21:26:15.873804 27814 ts_manager.cc:194] Registered new tserver with Master: e4be445cf919484e85f6bec392e31364 (127.14.142.1:39977)
I20260926 21:26:15.875113 27814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:60399
Sep 26 21:26:15 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457975, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
W20260926 21:26:16.030049 28160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:16.030742 28160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:16.030943 28160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:16.043247 28160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:16.043442 28160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:16.043640 28160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:26:16.066869 28160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.14.142.20:40033/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:38571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=42223
--tserver_master_addrs=127.14.142.62:41095
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:16.068989 28160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:16.070838 28160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:16.117358 28167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:16.118783 28166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:16.123154 28160 server_base.cc:1034] running on GCE node
W20260926 21:26:16.123538 28169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:16.124514 28160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:16.127219 28160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:16.128533 28160 hybrid_clock.cc:648] HybridClock initialized: now 1790457976128470 us; error 55 us; skew 500 ppm
I20260926 21:26:16.129021 28160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:16 dist-test-slave-4d5t krb5kdc[26803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457976, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:16.133128 28160 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:26:16.135059 28160 webserver.cc:460] Webserver started at http://127.14.142.3:42223/ using document root <none> and password file <none>
I20260926 21:26:16.135788 28160 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:16.135946 28160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 21:26:16 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457976, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:16.163892 28160 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260926 21:26:16.168751 28177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:16.170723 28160 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 21:26:16.170943 28160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "01fd217eb49545e09201749506160c8c"
format_stamp: "Formatted at 2026-09-26 21:26:12 on dist-test-slave-4d5t"
server_key: "2388dd1ec2fe3485c8ccca8488913293"
server_key_iv: "08192e769b06321c58813bddc65ed8b9"
server_key_version: "kuduclusterkey@0"
I20260926 21:26:16.172688 28160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:16.199754 28160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:16.206092 28160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:16.206518 28160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:16.208062 28160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:16.210246 28160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:16.210388 28160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:16.210522 28160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:16.210619 28160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:16.272926 28160 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:38571
I20260926 21:26:16.272931 28289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:38571 every 8 connection(s)
I20260926 21:26:16.275398 28160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:26:16.280287 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28160
I20260926 21:26:16.315231 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.281373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.14.142.62:41095)
0926 21:26:16.291753 (+ 10380us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.291758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.291778 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:26:16.291835 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.291839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.291967 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.292090 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:16.294591 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.295813 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.298122 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.298877 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.299032 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.303474 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:16.303481 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:16.303487 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:16.303554 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:16.307522 (+ 3968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.308628 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.308635 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.308639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.308811 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.312238 (+ 3427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.312245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.312248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.312760 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:16.313066 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.314496 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.314782 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10271,"thread_start_us":15201,"threads_started":1}
Sep 26 21:26:16 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457976, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:16.337105 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.296805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:54037 (local address 127.14.142.62:41095)
0926 21:26:16.301589 (+ 4784us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.301598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.302472 (+ 874us) server_negotiation.cc:407] Connection header received
0926 21:26:16.311671 (+ 9199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.311677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.311785 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.311893 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:16.316195 (+ 4302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.317534 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.321319 (+ 3785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.322091 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.322212 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.327532 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:16.327541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:16.327547 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:16.327597 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:16.332114 (+ 4517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.333719 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.333726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.333730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.333936 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.334594 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.334600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.334604 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.335026 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:16.335228 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.336351 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.336650 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4581,"thread_start_us":4795,"threads_started":1}
I20260926 21:26:16.339973 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.292905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41095 (local address 127.14.142.3:54037)
0926 21:26:16.302304 (+ 9399us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:16.302361 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:16.311291 (+ 8930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:16.312409 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:16.312423 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:16.313132 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:16.315916 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.315955 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.317996 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.318003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.321092 (+ 3089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.321128 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.322662 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.322672 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.323037 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.324075 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:16.324111 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:16.327144 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:16.332492 (+ 5348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:16.332504 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:16.332528 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:16.333557 (+ 1029us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:16.334104 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:16.334109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:16.334113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:16.334443 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:16.335329 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:16.335340 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:16.335841 (+ 501us) client_negotiation.cc:769] Sending connection context
0926 21:26:16.337580 (+ 1739us) client_negotiation.cc:240] Negotiation successful
0926 21:26:16.338272 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9083,"thread_start_us":8851,"threads_started":1}
I20260926 21:26:16.342185 28290 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41095
I20260926 21:26:16.342552 28290 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:16.343480 28290 heartbeater.cc:511] Master 127.14.142.62:41095 requested a full tablet report, sending...
I20260926 21:26:16.346141 27814 ts_manager.cc:194] Registered new tserver with Master: 01fd217eb49545e09201749506160c8c (127.14.142.3:38571)
I20260926 21:26:16.347295 27814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:54037
I20260926 21:26:16.348707 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 21:26:16 dist-test-slave-4d5t krb5kdc[26803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457975, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for HTTP/127.14.142.20@KRBTEST.COM
I20260926 21:26:16.389447 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.371954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.14.142.62:41095)
0926 21:26:16.372152 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.372158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.372491 (+ 333us) server_negotiation.cc:407] Connection header received
0926 21:26:16.372844 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.372849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.372935 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.373052 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:16.375136 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.376281 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.378267 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.378836 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.378979 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.382474 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:16.382480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:16.382486 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:16.382539 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:16.386054 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.386981 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.386988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.386992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.387142 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.387612 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.387618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.387622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.388004 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:16.388235 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.388836 (+ 601us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.389092 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 21:26:16.397392 27814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:26:16.408836 27814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:26:16.458037 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.429263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:38571 (local address 127.0.0.1:44680)
0926 21:26:16.439071 (+ 9808us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:16.439104 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:16.439275 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:16.449954 (+ 10679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:16.449960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:16.450006 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:16.450504 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.450520 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.452340 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.452345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.454864 (+ 2519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.454879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.456953 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.456960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.457172 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.457232 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 21:26:16.457352 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 21:26:16.457556 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9627,"mutex_wait_us":40,"thread_start_us":3523,"threads_started":1}
I20260926 21:26:16.459772 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.430326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.14.142.3:38571)
0926 21:26:16.448110 (+ 17784us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.448123 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.448167 (+ 44us) server_negotiation.cc:407] Connection header received
0926 21:26:16.448302 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.448315 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.449752 (+ 1437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.449958 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:16.450709 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.452170 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.455090 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.456699 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.458870 (+ 2171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.459147 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.459228 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.459405 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17604,"thread_start_us":17455,"threads_started":1}
I20260926 21:26:16.459803 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.429020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33401 (local address 127.0.0.1:35340)
0926 21:26:16.433267 (+ 4247us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:16.433324 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:16.433545 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:16.449563 (+ 16018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:16.449573 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:16.449611 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:16.450120 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.450140 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.453610 (+ 3470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.453616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.455813 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.455828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.459014 (+ 3186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.459019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.459200 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.459236 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:26:16.459300 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 21:26:16.459427 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3911,"mutex_wait_us":72,"thread_start_us":10983,"threads_started":1}
I20260926 21:26:16.460924 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.429885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35340 (local address 127.14.142.2:33401)
0926 21:26:16.448893 (+ 19008us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.448901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.448928 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:26:16.449021 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.449027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.449277 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.450248 (+ 971us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:16.450476 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.453411 (+ 2935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.456006 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.457427 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.460143 (+ 2716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.460344 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.460417 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.460630 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18863,"thread_start_us":18748,"threads_started":1}
I20260926 21:26:16.463609 28225 tablet_service.cc:1467] Processing CreateTablet for tablet 836bdf42b4ba455d9294a1ad1aa732ec (DEFAULT_TABLE table=test-table [id=cd5ae3a142344ef2a27982daf3db3db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:16.466393 28225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836bdf42b4ba455d9294a1ad1aa732ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:16.468612 28091 tablet_service.cc:1467] Processing CreateTablet for tablet 836bdf42b4ba455d9294a1ad1aa732ec (DEFAULT_TABLE table=test-table [id=cd5ae3a142344ef2a27982daf3db3db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:16.470196 28091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836bdf42b4ba455d9294a1ad1aa732ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:16.473132 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.441668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39977 (local address 127.0.0.1:58604)
0926 21:26:16.446483 (+ 4815us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:16.446514 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:16.446663 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:16.450707 (+ 4044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:16.450712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:16.450739 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:16.451129 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.451143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.460698 (+ 9555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.460704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.462754 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.462771 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.465053 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.465060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.465243 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.465290 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 21:26:16.465368 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 21:26:16.465509 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4654,"thread_start_us":4463,"threads_started":1}
I20260926 21:26:16.480469 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.441874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.14.142.1:39977)
0926 21:26:16.449040 (+ 7166us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.449049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.449080 (+ 31us) server_negotiation.cc:407] Connection header received
0926 21:26:16.449196 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.449203 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.449488 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.449786 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:16.452856 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.454656 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.463029 (+ 8373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.464706 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.479408 (+ 14702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.479719 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.479817 (+ 98us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.480105 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6945,"thread_start_us":6944,"threads_started":1}
I20260926 21:26:16.486131 27946 tablet_service.cc:1467] Processing CreateTablet for tablet 836bdf42b4ba455d9294a1ad1aa732ec (DEFAULT_TABLE table=test-table [id=cd5ae3a142344ef2a27982daf3db3db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:16.487914 27946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 836bdf42b4ba455d9294a1ad1aa732ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:16.505103 28312 tablet_bootstrap.cc:492] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Bootstrap starting.
I20260926 21:26:16.518467 28312 tablet_bootstrap.cc:654] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:16.525734 28312 log.cc:824] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:16.541651 28313 tablet_bootstrap.cc:492] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Bootstrap starting.
I20260926 21:26:16.543680 28312 tablet_bootstrap.cc:492] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: No bootstrap required, opened a new log
I20260926 21:26:16.543995 28312 ts_tablet_manager.cc:1397] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.000s
I20260926 21:26:16.545737 28313 tablet_bootstrap.cc:654] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:16.549177 28315 tablet_bootstrap.cc:492] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Bootstrap starting.
I20260926 21:26:16.564268 28313 log.cc:824] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:16.565979 28315 tablet_bootstrap.cc:654] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:16.567382 28315 log.cc:824] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:16.567458 28312 raft_consensus.cc:348] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.567957 28312 raft_consensus.cc:374] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:16.568068 28312 raft_consensus.cc:729] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01fd217eb49545e09201749506160c8c, State: Initialized, Role: FOLLOWER
I20260926 21:26:16.568861 28312 consensus_queue.cc:260] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.576176 28313 tablet_bootstrap.cc:492] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: No bootstrap required, opened a new log
I20260926 21:26:16.578943 28313 ts_tablet_manager.cc:1397] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.000s
I20260926 21:26:16.579007 28290 heartbeater.cc:503] Master 127.14.142.62:41095 was elected leader, sending a full tablet report...
I20260926 21:26:16.580008 28312 ts_tablet_manager.cc:1428] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Time spent starting tablet: real 0.036s user 0.020s sys 0.000s
I20260926 21:26:16.590806 28315 tablet_bootstrap.cc:492] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: No bootstrap required, opened a new log
I20260926 21:26:16.591182 28315 ts_tablet_manager.cc:1397] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.003s
I20260926 21:26:16.591701 28156 heartbeater.cc:503] Master 127.14.142.62:41095 was elected leader, sending a full tablet report...
I20260926 21:26:16.616034 28313 raft_consensus.cc:348] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.619704 28315 raft_consensus.cc:348] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.620244 28315 raft_consensus.cc:374] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:16.620347 28315 raft_consensus.cc:729] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4be445cf919484e85f6bec392e31364, State: Initialized, Role: FOLLOWER
I20260926 21:26:16.621199 28315 consensus_queue.cc:260] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.622448 28313 raft_consensus.cc:374] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:16.622583 28313 raft_consensus.cc:729] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155160e9a7a241b78dceb8f3c9f8d53e, State: Initialized, Role: FOLLOWER
I20260926 21:26:16.623991 28313 consensus_queue.cc:260] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.631552 28313 ts_tablet_manager.cc:1428] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Time spent starting tablet: real 0.052s user 0.009s sys 0.007s
I20260926 21:26:16.633337 28012 heartbeater.cc:503] Master 127.14.142.62:41095 was elected leader, sending a full tablet report...
I20260926 21:26:16.635215 28315 ts_tablet_manager.cc:1428] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Time spent starting tablet: real 0.044s user 0.017s sys 0.005s
I20260926 21:26:16.696671 28318 raft_consensus.cc:482] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:26:16.697016 28318 raft_consensus.cc:504] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.699047 28318 leader_election.cc:288] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 155160e9a7a241b78dceb8f3c9f8d53e (127.14.142.2:33401), e4be445cf919484e85f6bec392e31364 (127.14.142.1:39977)
I20260926 21:26:16.709172 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.699602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33401 (local address 127.14.142.3:57871)
0926 21:26:16.701735 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:16.701756 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:16.701890 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:16.702417 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:16.702423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:16.702452 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:16.702652 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.702661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.704517 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.704523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.706616 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.706631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.708322 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.708328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.708602 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.708637 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 21:26:16.708729 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 21:26:16.708871 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":1}
I20260926 21:26:16.709914 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.700891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:57871 (local address 127.14.142.2:33401)
0926 21:26:16.701357 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.701363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.702005 (+ 642us) server_negotiation.cc:407] Connection header received
0926 21:26:16.702087 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.702092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.702176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.702252 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:16.702807 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.703856 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.706812 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.708013 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.708160 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.708236 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.709532 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.709677 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260926 21:26:16.711277 28111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bdf42b4ba455d9294a1ad1aa732ec" candidate_uuid: "01fd217eb49545e09201749506160c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" is_pre_election: true
I20260926 21:26:16.711869 28111 raft_consensus.cc:2454] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01fd217eb49545e09201749506160c8c in term 0.
I20260926 21:26:16.712667 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.700127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:39977 (local address 127.14.142.3:57009)
0926 21:26:16.702706 (+ 2579us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:16.702725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:16.702858 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:16.705895 (+ 3037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:16.705900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:16.705932 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:16.706391 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.706404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.707893 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.707899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.710015 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:16.710030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.711788 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.711794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:16.711972 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.712008 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:26:16.712090 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 21:26:16.712228 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2452,"mutex_wait_us":76,"thread_start_us":5493,"threads_started":1}
I20260926 21:26:16.712788 28178 leader_election.cc:302] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01fd217eb49545e09201749506160c8c, 155160e9a7a241b78dceb8f3c9f8d53e; no voters:
I20260926 21:26:16.713299 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.701151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:57009 (local address 127.14.142.1:39977)
0926 21:26:16.701512 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.701518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.704005 (+ 2487us) server_negotiation.cc:407] Connection header received
0926 21:26:16.704111 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.704116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.704208 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.704293 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:16.706617 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.707705 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.710209 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.711416 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.711582 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.711657 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.712935 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.713079 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260926 21:26:16.713416 28318 raft_consensus.cc:2781] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:26:16.713555 28318 raft_consensus.cc:482] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:26:16.713641 28318 raft_consensus.cc:3037] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:16.714389 27967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bdf42b4ba455d9294a1ad1aa732ec" candidate_uuid: "01fd217eb49545e09201749506160c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4be445cf919484e85f6bec392e31364" is_pre_election: true
I20260926 21:26:16.714943 27967 raft_consensus.cc:2454] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01fd217eb49545e09201749506160c8c in term 0.
I20260926 21:26:16.716776 28318 raft_consensus.cc:504] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.717895 27967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bdf42b4ba455d9294a1ad1aa732ec" candidate_uuid: "01fd217eb49545e09201749506160c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4be445cf919484e85f6bec392e31364"
I20260926 21:26:16.717921 28111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836bdf42b4ba455d9294a1ad1aa732ec" candidate_uuid: "01fd217eb49545e09201749506160c8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "155160e9a7a241b78dceb8f3c9f8d53e"
I20260926 21:26:16.718093 27967 raft_consensus.cc:3037] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:16.718122 28111 raft_consensus.cc:3037] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:16.721238 27967 raft_consensus.cc:2454] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01fd217eb49545e09201749506160c8c in term 1.
I20260926 21:26:16.721240 28111 raft_consensus.cc:2454] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01fd217eb49545e09201749506160c8c in term 1.
I20260926 21:26:16.721761 28178 leader_election.cc:302] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01fd217eb49545e09201749506160c8c, 155160e9a7a241b78dceb8f3c9f8d53e; no voters:
I20260926 21:26:16.722090 28318 leader_election.cc:288] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [CANDIDATE]: Term 1 election: Requested vote from peers 155160e9a7a241b78dceb8f3c9f8d53e (127.14.142.2:33401), e4be445cf919484e85f6bec392e31364 (127.14.142.1:39977)
I20260926 21:26:16.722357 28318 raft_consensus.cc:2781] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:16.722618 28318 raft_consensus.cc:686] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 1 LEADER]: Becoming Leader. State: Replica: 01fd217eb49545e09201749506160c8c, State: Running, Role: LEADER
I20260926 21:26:16.723196 28318 consensus_queue.cc:237] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } }
I20260926 21:26:16.735427 27813 catalog_manager.cc:5495] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 01fd217eb49545e09201749506160c8c (127.14.142.3). New cstate: current_term: 1 leader_uuid: "01fd217eb49545e09201749506160c8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01fd217eb49545e09201749506160c8c" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 } health_report { overall_health: UNKNOWN } } }
W20260926 21:26:16.791363 28293 tablet.cc:2369] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:26:16.799788 28013 tablet.cc:2369] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:26:16.802659 28157 tablet.cc:2369] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:16.815157 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.809813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.14.142.3:38571)
0926 21:26:16.810015 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.810020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.810034 (+ 14us) server_negotiation.cc:407] Connection header received
0926 21:26:16.810120 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.810123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.810202 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.810362 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:26:16.810753 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.811491 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.812999 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.813394 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.813536 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.814047 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:26:16.814579 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:26:16.814735 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.814794 (+ 59us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.814877 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:26:16.827761 27967 raft_consensus.cc:1261] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 1 FOLLOWER]: Refusing update from remote peer 01fd217eb49545e09201749506160c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:16.827749 28111 raft_consensus.cc:1261] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 1 FOLLOWER]: Refusing update from remote peer 01fd217eb49545e09201749506160c8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:16.828892 28329 consensus_queue.cc:1035] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4be445cf919484e85f6bec392e31364" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 39977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:16.829325 28318 consensus_queue.cc:1035] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "155160e9a7a241b78dceb8f3c9f8d53e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:16.855135 28333 mvcc.cc:205] Tried to move back new op lower bound from 7333715873075154944 to 7333715872664190976. Current Snapshot: MvccSnapshot[applied={T|T < 7333715873075154944}]
I20260926 21:26:16.885138 28334 mvcc.cc:205] Tried to move back new op lower bound from 7333715873075154944 to 7333715872664190976. Current Snapshot: MvccSnapshot[applied={T|T < 7333715873075154944}]
I20260926 21:26:16.891965 27813 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:16.892266 27813 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:16.896492 27813 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c1ee0e6fef254260b9c3b4acb476b5eb: Sending DeleteTablet for 3 replicas of tablet 836bdf42b4ba455d9294a1ad1aa732ec
I20260926 21:26:16.897833 28091 tablet_service.cc:1514] Processing DeleteTablet for tablet 836bdf42b4ba455d9294a1ad1aa732ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:16 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:35340
I20260926 21:26:16.897908 28225 tablet_service.cc:1514] Processing DeleteTablet for tablet 836bdf42b4ba455d9294a1ad1aa732ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:16 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:44680
I20260926 21:26:16.898661 27946 tablet_service.cc:1514] Processing DeleteTablet for tablet 836bdf42b4ba455d9294a1ad1aa732ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:16 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:58604
I20260926 21:26:16.901840 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 27851
I20260926 21:26:16.904531 28343 tablet_replica.cc:330] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: stopping tablet replica
I20260926 21:26:16.905243 28343 raft_consensus.cc:2229] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:26:16.906167 28343 raft_consensus.cc:2258] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:16.907841 28344 tablet_replica.cc:330] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: stopping tablet replica
I20260926 21:26:16.908512 28344 raft_consensus.cc:2229] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:16.909324 28344 raft_consensus.cc:2258] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:16.910503 28342 tablet_replica.cc:330] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: stopping tablet replica
I20260926 21:26:16.912132 28344 ts_tablet_manager.cc:1905] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:16.912706 28342 raft_consensus.cc:2229] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:16.914316 28343 ts_tablet_manager.cc:1905] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:16.920121 28344 ts_tablet_manager.cc:1918] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:16.920783 28342 raft_consensus.cc:2258] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:16.923210 28342 ts_tablet_manager.cc:1905] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:16.930418 28342 ts_tablet_manager.cc:1918] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:16.930434 28343 ts_tablet_manager.cc:1918] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:16.930699 28344 log.cc:1196] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/836bdf42b4ba455d9294a1ad1aa732ec
I20260926 21:26:16.931375 28344 ts_tablet_manager.cc:1939] T 836bdf42b4ba455d9294a1ad1aa732ec P e4be445cf919484e85f6bec392e31364: Deleting consensus metadata
I20260926 21:26:16.933624 27801 catalog_manager.cc:4841] TS e4be445cf919484e85f6bec392e31364 (127.14.142.1:39977): tablet 836bdf42b4ba455d9294a1ad1aa732ec (table test-table [id=cd5ae3a142344ef2a27982daf3db3db4]) successfully deleted
I20260926 21:26:16.934067 28342 log.cc:1196] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/836bdf42b4ba455d9294a1ad1aa732ec
I20260926 21:26:16.934748 28342 ts_tablet_manager.cc:1939] T 836bdf42b4ba455d9294a1ad1aa732ec P 155160e9a7a241b78dceb8f3c9f8d53e: Deleting consensus metadata
I20260926 21:26:16.936690 27798 catalog_manager.cc:4841] TS 155160e9a7a241b78dceb8f3c9f8d53e (127.14.142.2:33401): tablet 836bdf42b4ba455d9294a1ad1aa732ec (table test-table [id=cd5ae3a142344ef2a27982daf3db3db4]) successfully deleted
I20260926 21:26:16.937072 28343 log.cc:1196] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/836bdf42b4ba455d9294a1ad1aa732ec
I20260926 21:26:16.937724 28343 ts_tablet_manager.cc:1939] T 836bdf42b4ba455d9294a1ad1aa732ec P 01fd217eb49545e09201749506160c8c: Deleting consensus metadata
I20260926 21:26:16.939482 28311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:16.902783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.14.142.1:39977)
0926 21:26:16.903112 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:16.903118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:16.907158 (+ 4040us) server_negotiation.cc:407] Connection header received
0926 21:26:16.907352 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:16.907360 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:16.907495 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:16.907625 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:16.919289 (+ 11664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.920739 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.922777 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:16.923428 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:16.923528 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:16.930793 (+ 7265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:16.930807 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:16.930832 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:16.930897 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:16.935184 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.936375 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.936529 (+ 154us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.936533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.936740 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:16.937305 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:16.937313 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:16.937318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:16.937881 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:16.938169 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:16.938823 (+ 654us) server_negotiation.cc:299] Negotiation successful
0926 21:26:16.939072 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260926 21:26:16.939574 27800 catalog_manager.cc:4841] TS 01fd217eb49545e09201749506160c8c (127.14.142.3:38571): tablet 836bdf42b4ba455d9294a1ad1aa732ec (table test-table [id=cd5ae3a142344ef2a27982daf3db3db4]) successfully deleted
I20260926 21:26:16.941126 28007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:17.040444 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 27851
I20260926 21:26:17.071272 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28016
I20260926 21:26:17.086308 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:17.072123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.14.142.2:33401)
0926 21:26:17.072417 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:17.072423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:17.072446 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:26:17.072684 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:17.072690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:17.072794 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:17.072894 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:17.074350 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:17.075105 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:17.076442 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:17.076933 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:17.077087 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:17.079859 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:17.079925 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:17.079948 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:17.079998 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:17.083118 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:17.083982 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:17.083992 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:17.083995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:17.084168 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:17.084604 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:17.084609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:17.084613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:17.084994 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:17.085304 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:17.085636 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 21:26:17.085869 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260926 21:26:17.087231 28151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:17.183869 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28016
I20260926 21:26:17.206184 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28160
I20260926 21:26:17.221769 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:17.206953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.14.142.3:38571)
0926 21:26:17.207431 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:17.207440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:17.207455 (+ 15us) server_negotiation.cc:407] Connection header received
0926 21:26:17.207506 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:17.207510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:17.207566 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:17.207695 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:17.209157 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:17.210011 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:17.211348 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:17.211828 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:17.211984 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:17.214854 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:17.214870 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:17.214887 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:17.214922 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:17.218245 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:17.219036 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:17.219043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:17.219045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:17.219230 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:17.219622 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:17.219627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:17.219630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:17.220006 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:17.220348 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:17.220722 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 21:26:17.221349 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383}
I20260926 21:26:17.222474 28285 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:17.342545 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28160
I20260926 21:26:17.360921 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 27781
I20260926 21:26:17.362223 27844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:17.480466 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 27781
I20260926 21:26:17.509929 14904 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 21:26:17.856889 14904 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 21:26:17.857117 14904 mini_ranger.cc:66] Stopping Ranger...
I20260926 21:26:18.203867 14904 mini_ranger.cc:68] Stopped Ranger
2026-09-26 21:26:18.204 UTC [26833] LOG: received smart shutdown request
2026-09-26 21:26:18.207 UTC [26833] LOG: background worker "logical replication launcher" (PID 26842) exited with exit code 1
2026-09-26 21:26:18.208 UTC [26837] LOG: shutting down
2026-09-26 21:26:18.403 UTC [26833] LOG: database system is shut down
2026-09-26T21:26:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56765 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:26:18 dist-test-slave-4d5t krb5kdc[28358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:26:18 dist-test-slave-4d5t krb5kdc[28358](info): set up 2 sockets
Sep 26 21:26:18 dist-test-slave-4d5t krb5kdc[28358](info): commencing operation
krb5kdc: starting...
W20260926 21:26:19.717312 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:26:19 dist-test-slave-4d5t krb5kdc[28358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:26:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:26:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:26:19.981508 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44981
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:33195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44981,127.14.142.61:44845,127.14.142.60:40887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:20.344462 28371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:20.344877 28371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:20.345009 28371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:20.352707 28371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:20.352837 28371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:20.352885 28371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:20.353035 28371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:20.353111 28371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:20.363611 28371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:33195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44981,127.14.142.61:44845,127.14.142.60:40887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:20.365162 28371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:20.366503 28371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:20.379179 28377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:20.379256 28379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:20.379154 28376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:20.380170 28371 server_base.cc:1034] running on GCE node
I20260926 21:26:20.381116 28371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:20.382650 28371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:20.383944 28371 hybrid_clock.cc:648] HybridClock initialized: now 1790457980383862 us; error 89 us; skew 500 ppm
I20260926 21:26:20.384387 28371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:20 dist-test-slave-4d5t krb5kdc[28358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:20.392916 28371 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:20.394814 28371 webserver.cc:460] Webserver started at http://127.14.142.62:43999/ using document root <none> and password file <none>
I20260926 21:26:20.395642 28371 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:20.395810 28371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:20.396163 28371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:20.400369 28371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "4bd1b3b3ec5c4b879b66560a863c2faf"
format_stamp: "Formatted at 2026-09-26 21:26:20 on dist-test-slave-4d5t"
server_key: "66fbd1c00fcf399d467e72351fe55e0d"
server_key_iv: "1763679f39ef780fcecf7f758f091bc4"
server_key_version: "encryptionkey@0"
I20260926 21:26:20.401610 28371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "4bd1b3b3ec5c4b879b66560a863c2faf"
format_stamp: "Formatted at 2026-09-26 21:26:20 on dist-test-slave-4d5t"
server_key: "66fbd1c00fcf399d467e72351fe55e0d"
server_key_iv: "1763679f39ef780fcecf7f758f091bc4"
server_key_version: "encryptionkey@0"
I20260926 21:26:20.408447 28371 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 21:26:20.412822 28386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:20.414597 28371 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:26:20.414852 28371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "4bd1b3b3ec5c4b879b66560a863c2faf"
format_stamp: "Formatted at 2026-09-26 21:26:20 on dist-test-slave-4d5t"
server_key: "66fbd1c00fcf399d467e72351fe55e0d"
server_key_iv: "1763679f39ef780fcecf7f758f091bc4"
server_key_version: "encryptionkey@0"
I20260926 21:26:20.415108 28371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:20.431232 28371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:20.435923 28371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:20.436241 28371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:20.456370 28371 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:44981
I20260926 21:26:20.456409 28437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:44981 every 8 connection(s)
I20260926 21:26:20.457898 28371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:20.462276 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28371
I20260926 21:26:20.462525 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:26:20.462610 28438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:20.463409 14904 external_mini_cluster.cc:1428] Setting key 4cd1fbea25e513b76c54581f35cf7427
I20260926 21:26:20.473555 28438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.14.142.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.61@KRBTEST.COM" created.
Sep 26 21:26:20 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:20.504716 28442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.477151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.14.142.62:44981)
0926 21:26:20.479091 (+ 1940us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:20.479105 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:20.479170 (+ 65us) server_negotiation.cc:407] Connection header received
0926 21:26:20.480938 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:20.480949 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:20.481279 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:20.481532 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:20.482897 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:20.485073 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:20.486781 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:20.487819 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:20.488063 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:20.494260 (+ 6197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:20.494274 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:20.494299 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:20.494350 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:20.498908 (+ 4558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:20.500169 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:20.500183 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:20.500190 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:20.500470 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:20.501194 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:20.501198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:20.501201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:20.502043 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:20.502285 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:20.503099 (+ 814us) server_negotiation.cc:299] Negotiation successful
0926 21:26:20.503446 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":1710,"spinlock_wait_cycles":37120,"thread_start_us":1485,"threads_started":1}
I20260926 21:26:20.504767 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.477165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44981 (local address 127.0.0.1:48748)
0926 21:26:20.478330 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.478394 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:20.480452 (+ 2058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:20.481563 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:20.481569 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:20.482216 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:20.482585 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:20.482603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:20.485316 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:20.485321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:20.486609 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:20.486622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:20.488104 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:20.488110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:20.488244 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:20.489282 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:20.489329 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:20.493922 (+ 4593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:20.499220 (+ 5298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:20.499243 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:20.499259 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:20.499923 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:20.500709 (+ 786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:20.500719 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:20.500725 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:20.501016 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:20.502280 (+ 1264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:20.502293 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:20.502607 (+ 314us) client_negotiation.cc:769] Sending connection context
0926 21:26:20.502940 (+ 333us) client_negotiation.cc:240] Negotiation successful
0926 21:26:20.503656 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":128,"thread_start_us":530,"threads_started":1}
I20260926 21:26:20.508448 28438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:26:20.509872 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.509248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46408)
0926 21:26:20.509470 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.509598 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260926 21:26:20.510448 28388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
I20260926 21:26:20.512804 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.512180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46420)
0926 21:26:20.512451 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.512589 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260926 21:26:20.513197 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:44845: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab.
Entry for principal kudu/127.14.142.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab.
I20260926 21:26:20.529446 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:44845
--webserver_interface=127.14.142.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab
--principal=kudu/127.14.142.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:33195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44981,127.14.142.61:44845,127.14.142.60:40887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:20.562697 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } attempt: 1
I20260926 21:26:20.563963 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.563494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46434)
0926 21:26:20.563711 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.563823 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260926 21:26:20.565143 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.564791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46438)
0926 21:26:20.564947 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.565015 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 21:26:20.565328 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:44845: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
I20260926 21:26:20.633615 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } attempt: 2
I20260926 21:26:20.634792 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.634309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46452)
0926 21:26:20.634575 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.634658 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 21:26:20.635952 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.635548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46454)
0926 21:26:20.635721 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.635810 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260926 21:26:20.636148 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:44845: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
I20260926 21:26:20.727501 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } attempt: 3
I20260926 21:26:20.729152 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.728605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46466)
0926 21:26:20.728893 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.729005 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260926 21:26:20.730626 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.730216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46476)
0926 21:26:20.730389 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.730477 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 21:26:20.730937 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:44845: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
I20260926 21:26:20.874301 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } attempt: 4
I20260926 21:26:20.875942 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.875289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46488)
0926 21:26:20.875670 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.875774 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260926 21:26:20.877200 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:20.876786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46502)
0926 21:26:20.876951 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:20.877058 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 21:26:20.877439 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:44845: Network error: Client connection negotiation failed: client connection to 127.14.142.61:44845: connect: Connection refused (error 111)
W20260926 21:26:20.886960 28445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:20.887392 28445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:20.887586 28445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:20.896795 28445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:20.896944 28445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:20.897089 28445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:20.897150 28445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:20.897192 28445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:20.909592 28445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:33195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44981,127.14.142.61:44845,127.14.142.60:40887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab
--principal=kudu/127.14.142.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:44845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.142.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:20.911128 28445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:20.912508 28445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:20.926172 28453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:20.926124 28451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:20.926139 28450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:20.927107 28445 server_base.cc:1034] running on GCE node
I20260926 21:26:20.927976 28445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:20.929567 28445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:20.930902 28445 hybrid_clock.cc:648] HybridClock initialized: now 1790457980930794 us; error 51 us; skew 500 ppm
I20260926 21:26:20.931627 28445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:20 dist-test-slave-4d5t krb5kdc[28358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:20.935492 28445 init.cc:377] Logged in from keytab as kudu/127.14.142.61@KRBTEST.COM (short username kudu)
I20260926 21:26:20.937323 28445 webserver.cc:460] Webserver started at http://127.14.142.61:34053/ using document root <none> and password file <none>
I20260926 21:26:20.938519 28445 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:20.938804 28445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:20.939342 28445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:20.942922 28445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data/instance:
uuid: "040a4dc034eb44b682f482f12b8a37fc"
format_stamp: "Formatted at 2026-09-26 21:26:20 on dist-test-slave-4d5t"
server_key: "fe7cccc2358e3076bc3aaa33e59b5a8c"
server_key_iv: "56692f731cac9e4ee3fdbbbec0b0f177"
server_key_version: "encryptionkey@0"
I20260926 21:26:20.943863 28445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal/instance:
uuid: "040a4dc034eb44b682f482f12b8a37fc"
format_stamp: "Formatted at 2026-09-26 21:26:20 on dist-test-slave-4d5t"
server_key: "fe7cccc2358e3076bc3aaa33e59b5a8c"
server_key_iv: "56692f731cac9e4ee3fdbbbec0b0f177"
server_key_version: "encryptionkey@0"
I20260926 21:26:20.949599 28445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260926 21:26:20.953724 28460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:20.955516 28445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:26:20.955834 28445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal
uuid: "040a4dc034eb44b682f482f12b8a37fc"
format_stamp: "Formatted at 2026-09-26 21:26:20 on dist-test-slave-4d5t"
server_key: "fe7cccc2358e3076bc3aaa33e59b5a8c"
server_key_iv: "56692f731cac9e4ee3fdbbbec0b0f177"
server_key_version: "encryptionkey@0"
I20260926 21:26:20.956064 28445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:20.973569 28445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:20.978652 28445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:20.978965 28445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:21.001475 28445 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.61:44845
I20260926 21:26:21.001510 28511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.61:44845 every 8 connection(s)
I20260926 21:26:21.003007 28445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
I20260926 21:26:21.007679 28512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:21.009985 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28445
I20260926 21:26:21.010186 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-1/wal/instance
I20260926 21:26:21.010658 14904 external_mini_cluster.cc:1428] Setting key d456e6e81fa41a5c96108019cfb170a6
I20260926 21:26:21.014703 28512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.142.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.60@KRBTEST.COM" created.
I20260926 21:26:21.040697 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.017532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48760 (local address 127.14.142.62:44981)
0926 21:26:21.018150 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.018159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.019145 (+ 986us) server_negotiation.cc:407] Connection header received
0926 21:26:21.021200 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.021209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.021309 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.021447 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.023977 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.025382 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.027615 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.028371 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.028530 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.032527 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.032536 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.032541 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.032690 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.036637 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.037833 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.037837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.037840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.037991 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.038579 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.038583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.038585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.038899 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.039027 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.040068 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.040285 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":288,"threads_started":1}
I20260926 21:26:21.042140 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.018130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44981 (local address 127.0.0.1:48760)
0926 21:26:21.018962 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.019033 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.020912 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.021974 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.021987 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.022823 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.023674 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.023698 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.025661 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.025666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.027386 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.027413 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.028718 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.028725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.028970 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.029740 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.029766 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.032084 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.036851 (+ 4767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.036865 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.036889 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.037665 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.038123 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.038131 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.038137 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.038408 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.039052 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.039064 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.039613 (+ 549us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.039953 (+ 340us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.040548 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":357,"threads_started":1}
I20260926 21:26:21.044363 28512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:26:21.066839 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.045198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46514)
0926 21:26:21.045358 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.045373 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.045462 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.046910 (+ 1448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.046914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.047655 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.047831 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.047843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.049480 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.049487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.050666 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.050689 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.052607 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.052614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.052910 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.053746 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.053795 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.057875 (+ 4080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.063124 (+ 5249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.063132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.063137 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.063679 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.064393 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.064402 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.064409 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.064665 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.065502 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.065506 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.065687 (+ 181us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.065851 (+ 164us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.066379 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 21:26:21.066943 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.045482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.14.142.61:44845)
0926 21:26:21.046287 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.046296 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.046335 (+ 39us) server_negotiation.cc:407] Connection header received
0926 21:26:21.046408 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.046416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.046668 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.046928 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.048023 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.049324 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.050866 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.052080 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.052270 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.058263 (+ 5993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.058289 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.058311 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.058351 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.062929 (+ 4578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.063870 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.063888 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.063890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.064040 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.064795 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.064800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.064803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.065374 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.065630 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.066000 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.066532 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":236,"server-negotiator.queue_time_us":576,"thread_start_us":385,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab.
Entry for principal kudu/127.14.142.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab.
I20260926 21:26:21.071427 28512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:26:21.072252 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:40887
--webserver_interface=127.14.142.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab
--principal=kudu/127.14.142.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:33195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44981,127.14.142.61:44845,127.14.142.60:40887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:21.073266 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.072523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49404)
0926 21:26:21.072903 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.073068 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
W20260926 21:26:21.073578 28464 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.075574 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.075203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49410)
0926 21:26:21.075366 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.075453 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 21:26:21.075811 28512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.125160 28512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 1
I20260926 21:26:21.126526 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.125926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49424)
0926 21:26:21.126169 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.126309 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 21:26:21.127673 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.127214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49434)
0926 21:26:21.127417 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.127519 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260926 21:26:21.127955 28512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.176807 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } attempt: 5
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:26:21.193857 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.177787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46530)
0926 21:26:21.178162 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.178197 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.178413 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.179001 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.179006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.179578 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.179866 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.179882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.181062 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.181069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.182513 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.182530 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.183346 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.183353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.183564 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.184347 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.184381 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.187023 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.191282 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.191287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.191290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.191696 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.192165 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.192169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.192172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.192305 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.193071 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.193074 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.193217 (+ 143us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.193346 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.193590 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260926 21:26:21.194609 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.178142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.14.142.61:44845)
0926 21:26:21.178361 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.178366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.178391 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:26:21.178669 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.178676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.178787 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.178889 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.180091 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.180906 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.182708 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.183172 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.183273 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.187285 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.187294 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.187299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.187350 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.191103 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.191853 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.191859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.191863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.192029 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.192436 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.192444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.192450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.192920 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.193092 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.193528 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.193851 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:26:21.195883 28438 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:26:21.196179 28512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 2
I20260926 21:26:21.197067 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.196677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49448)
0926 21:26:21.196841 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.196927 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 21:26:21.197167 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.196678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49460)
0926 21:26:21.196899 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.196996 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260926 21:26:21.198101 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.197714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49474)
0926 21:26:21.197876 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.197976 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 21:26:21.198149 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.197736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49482)
0926 21:26:21.197945 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.198023 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260926 21:26:21.198356 28512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
W20260926 21:26:21.198376 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.249650 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 1
I20260926 21:26:21.250900 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.250418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49488)
0926 21:26:21.250632 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.250742 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 21:26:21.252308 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.251697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49500)
0926 21:26:21.251997 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.252123 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260926 21:26:21.252714 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.289649 28512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 3
I20260926 21:26:21.290940 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.290329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49508)
0926 21:26:21.290602 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.290729 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 21:26:21.292234 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.291619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49518)
0926 21:26:21.291923 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.292044 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260926 21:26:21.292526 28512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.320990 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 2
I20260926 21:26:21.322173 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.321662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49528)
0926 21:26:21.321909 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.322005 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 21:26:21.323153 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.322816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49540)
0926 21:26:21.322972 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.323040 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 21:26:21.323348 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.429595 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 3
I20260926 21:26:21.430778 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.430335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49554)
0926 21:26:21.430533 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.430619 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 21:26:21.431793 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.431414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49566)
0926 21:26:21.431625 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.431697 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260926 21:26:21.431994 28438 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
I20260926 21:26:21.435822 28512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 4
I20260926 21:26:21.436911 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.436354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49580)
0926 21:26:21.436632 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.436731 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 21:26:21.437938 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.437566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49582)
0926 21:26:21.437725 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.437810 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 21:26:21.438180 28512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:40887: Network error: Client connection negotiation failed: client connection to 127.14.142.60:40887: connect: Connection refused (error 111)
W20260926 21:26:21.451810 28520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:21.452224 28520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:21.452417 28520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:21.460624 28520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:21.460773 28520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:21.460903 28520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:21.461012 28520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:21.461087 28520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:21.473204 28520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:33195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44981,127.14.142.61:44845,127.14.142.60:40887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab
--principal=kudu/127.14.142.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:40887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.142.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:21.474673 28520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:21.476090 28520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:21.489250 28526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:21.489634 28528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:21.489279 28525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:21.490237 28520 server_base.cc:1034] running on GCE node
I20260926 21:26:21.491140 28520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:21.493396 28520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:21.494637 28520 hybrid_clock.cc:648] HybridClock initialized: now 1790457981494581 us; error 51 us; skew 500 ppm
I20260926 21:26:21.495047 28520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457981, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:21.498454 28520 init.cc:377] Logged in from keytab as kudu/127.14.142.60@KRBTEST.COM (short username kudu)
I20260926 21:26:21.500434 28520 webserver.cc:460] Webserver started at http://127.14.142.60:33945/ using document root <none> and password file <none>
I20260926 21:26:21.501261 28520 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:21.501398 28520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:21.501744 28520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:21.505069 28520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data/instance:
uuid: "627e7d4cc542450a874f1939518b1ce5"
format_stamp: "Formatted at 2026-09-26 21:26:21 on dist-test-slave-4d5t"
server_key: "5f77af1e55ac73cb9027ccedaffd08b1"
server_key_iv: "30899d6c01c1f2b7596f398502f434a6"
server_key_version: "encryptionkey@0"
I20260926 21:26:21.505971 28520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal/instance:
uuid: "627e7d4cc542450a874f1939518b1ce5"
format_stamp: "Formatted at 2026-09-26 21:26:21 on dist-test-slave-4d5t"
server_key: "5f77af1e55ac73cb9027ccedaffd08b1"
server_key_iv: "30899d6c01c1f2b7596f398502f434a6"
server_key_version: "encryptionkey@0"
I20260926 21:26:21.512876 28520 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:26:21.516857 28535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:21.518352 28520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:26:21.518599 28520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal
uuid: "627e7d4cc542450a874f1939518b1ce5"
format_stamp: "Formatted at 2026-09-26 21:26:21 on dist-test-slave-4d5t"
server_key: "5f77af1e55ac73cb9027ccedaffd08b1"
server_key_iv: "30899d6c01c1f2b7596f398502f434a6"
server_key_version: "encryptionkey@0"
I20260926 21:26:21.518848 28520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:21.539840 28520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:21.544718 28520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:21.545017 28520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:21.565402 28520 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.60:40887
I20260926 21:26:21.565434 28586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.60:40887 every 8 connection(s)
I20260926 21:26:21.566905 28520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
I20260926 21:26:21.571834 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:21.574398 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28520
I20260926 21:26:21.574622 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790457761955429-14904-0/minicluster-data/master-2/wal/instance
I20260926 21:26:21.575213 14904 external_mini_cluster.cc:1428] Setting key 755d85347f8659e1ba0de6c785d7229b
I20260926 21:26:21.580443 28587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457981, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:21.599968 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.583706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.14.142.62:44981)
0926 21:26:21.584250 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.584257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.584913 (+ 656us) server_negotiation.cc:407] Connection header received
0926 21:26:21.585117 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.585125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.585229 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.585360 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.586523 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.587428 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.589339 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.589752 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.589871 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.593888 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.593894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.593897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.593941 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.596530 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.597329 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.597337 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.597340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.597490 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.597917 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.597925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.597929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.598213 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.598335 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.599251 (+ 916us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.599499 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":276,"threads_started":1}
I20260926 21:26:21.601408 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.577466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.14.142.60:40887)
0926 21:26:21.578387 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.578400 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.578462 (+ 62us) server_negotiation.cc:407] Connection header received
0926 21:26:21.580150 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.580174 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.580662 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.581034 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.582990 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.585005 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.587008 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.587979 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.588126 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.593337 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.593349 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.593368 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.593410 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.596613 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.597381 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.597388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.597390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.597561 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.597991 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.597995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.597997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.598777 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.599090 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.599667 (+ 577us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.599990 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":692,"thread_start_us":442,"threads_started":1}
I20260926 21:26:21.601449 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.583744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44981 (local address 127.0.0.1:48770)
0926 21:26:21.584761 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.584814 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.584978 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.585420 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.585428 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.586066 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.586358 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.586372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.587619 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.587624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.589167 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.589183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.590017 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.590027 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.590261 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.591128 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.591164 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.593634 (+ 2470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.596733 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.596743 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.596760 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.597183 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.597602 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.597609 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.597613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.597793 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.598364 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.598372 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.598806 (+ 434us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.599092 (+ 286us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.599864 (+ 772us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":44,"spinlock_wait_cycles":22272,"thread_start_us":559,"threads_started":1}
I20260926 21:26:21.603129 28587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:26:21.609187 28438 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 4
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457981, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:26:21.622594 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.604228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:44845 (local address 127.0.0.1:46544)
0926 21:26:21.604511 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.604534 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.604682 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.605215 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.605219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.605641 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.606054 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.606070 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.607165 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.607171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.608169 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.608181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.609068 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.609073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.609890 (+ 817us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.611862 (+ 1972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.611899 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.615229 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.619438 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.619445 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.619449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.620073 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.620574 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.620579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.620583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.620788 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.621562 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.621567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.621717 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.621858 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.622233 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260926 21:26:21.622617 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.604402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.14.142.61:44845)
0926 21:26:21.604717 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.604726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.604751 (+ 25us) server_negotiation.cc:407] Connection header received
0926 21:26:21.604854 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.604860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.605005 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.605137 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.606253 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.607004 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.608319 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.608884 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.609052 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.615509 (+ 6457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.615516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.615521 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.615572 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.619292 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.620254 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.620261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.620264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.620418 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.620927 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.620933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.620938 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.621303 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.621449 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.621970 (+ 521us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.622204 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260926 21:26:21.623826 28587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:26:21.624776 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.605873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.14.142.62:44981)
0926 21:26:21.606113 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.606121 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.606147 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:26:21.606354 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.606360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.606452 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.606596 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.607907 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.608717 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.610244 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.610938 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.611040 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.616538 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.616545 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.616549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.616642 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.620222 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.621296 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.621301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.621305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.621447 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.622821 (+ 1374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.622825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.622828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.623145 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.623631 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.624205 (+ 574us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.624433 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 21:26:21.626854 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.610182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49592 (local address 127.14.142.60:40887)
0926 21:26:21.610382 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.610388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.610405 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:26:21.610458 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.610463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.610537 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.610655 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.611765 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.612535 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.613895 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.614309 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.615013 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.618915 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.618921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.618929 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.618975 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.622631 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.623798 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.623805 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.623813 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.623968 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.624500 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.624507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.624512 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.625731 (+ 1219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.625929 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.626306 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.626530 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 21:26:21.626878 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.609910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49592)
0926 21:26:21.610211 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.610240 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.610350 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.610727 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.610732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.611295 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.611538 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.611561 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.612763 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.612770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.613734 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.613747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.614526 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.614531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.614662 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.615427 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.615460 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.618706 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.623185 (+ 4479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.623190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.623193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.623702 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.624124 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.624129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.624133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.624324 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.625890 (+ 1566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.625895 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.626069 (+ 174us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.626217 (+ 148us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.626503 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457981, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:26:21.634387 28438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: Bootstrap starting.
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:26:21.639004 28438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:21.640641 28438 log.cc:824] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:21.641304 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.625283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49598)
0926 21:26:21.625464 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.625490 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.625595 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.627118 (+ 1523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.627122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.627579 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.627723 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.627731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.629181 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.629185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.630552 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.630570 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.631476 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.631491 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.631685 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.632662 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.632688 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.634894 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.637852 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.637858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.637861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.638388 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.638807 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.638814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.638820 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.638986 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.639911 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.639915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.640093 (+ 178us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.640252 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.640885 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 21:26:21.641667 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.624762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.14.142.60:40887)
0926 21:26:21.625534 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.625539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.625559 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:26:21.625711 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.625716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.625780 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.625881 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.627885 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.629031 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.630707 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.631335 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.631467 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.635118 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.635122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.635126 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.635153 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.637703 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.638525 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.638533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.638537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.638680 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.639130 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.639136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.639139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.639673 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.639856 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.640738 (+ 882us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.641065 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":659,"thread_start_us":639,"threads_started":1}
I20260926 21:26:21.644029 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.626551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.14.142.61:44845)
0926 21:26:21.626721 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.626725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.626932 (+ 207us) server_negotiation.cc:407] Connection header received
0926 21:26:21.627142 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.627152 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.627212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.627266 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.629092 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.629817 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.631236 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.631899 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.632037 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.637801 (+ 5764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.637808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.637813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.637857 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.640506 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.641551 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.641557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.641561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.641743 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.642238 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.642246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.642253 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.642677 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.642968 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.643424 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.643696 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 21:26:21.645342 28438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: No bootstrap required, opened a new log
I20260926 21:26:21.646930 28587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: Bootstrap starting.
I20260926 21:26:21.649645 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:26:21.650595 28587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:21.651893 28587 log.cc:824] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:21.655076 28438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.655424 28438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:21.655580 28438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bd1b3b3ec5c4b879b66560a863c2faf, State: Initialized, Role: FOLLOWER
I20260926 21:26:21.656219 28438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.656757 28587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: No bootstrap required, opened a new log
I20260926 21:26:21.671015 28598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:21.671706 28598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:21.672461 28587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.673010 28587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:21.673198 28587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627e7d4cc542450a874f1939518b1ce5, State: Initialized, Role: FOLLOWER
I20260926 21:26:21.673233 28438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:21.674108 28587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
W20260926 21:26:21.689601 28392 tablet.cc:2369] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:21.693041 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.659165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.14.142.60:40887)
0926 21:26:21.659393 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.659400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.666467 (+ 7067us) server_negotiation.cc:407] Connection header received
0926 21:26:21.666991 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.666998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.667085 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.667167 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.669803 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.670888 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.673230 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.673916 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.674035 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.679213 (+ 5178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.679223 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.679232 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.679301 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.685235 (+ 5934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.687105 (+ 1870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.687116 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.687123 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.687373 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.687946 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.687954 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.687960 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.688539 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.688934 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.691773 (+ 2839us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.692407 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 21:26:21.695544 28604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:21.696451 28604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:21.697036 28587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:21.699682 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.659961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.14.142.62:44981)
0926 21:26:21.660263 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.660273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.663142 (+ 2869us) server_negotiation.cc:407] Connection header received
0926 21:26:21.664479 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.664489 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.664674 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.664790 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.675231 (+ 10441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.676732 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.680326 (+ 3594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.681074 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.681191 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.688773 (+ 7582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.688780 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.688785 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.688833 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.694681 (+ 5848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.696441 (+ 1760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.696451 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.696456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.696662 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.697251 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.697256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.697259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.697707 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.697997 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.698717 (+ 720us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.699086 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260926 21:26:21.704731 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.663578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46558 (local address 127.14.142.61:44845)
0926 21:26:21.663815 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.663821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.680116 (+ 16295us) server_negotiation.cc:407] Connection header received
0926 21:26:21.680288 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.680293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.680381 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.680472 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.682521 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.683952 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.687262 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.688212 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.688353 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.692541 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.692571 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.692576 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.692634 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.698105 (+ 5471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.699298 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.699303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.699306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.699474 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.700161 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.700169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.700175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.702757 (+ 2582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.703047 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.703762 (+ 715us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.704154 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 21:26:21.718158 28623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:26:21.718384 28623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:26:21.720521 28520 master_runner.cc:186] Error getting master registration for 127.14.142.61:44845: OK, instance_id { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" instance_seqno: 1790457980994906 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 21:26:21.732049 28520 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 21:26:21.733155 28626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:26:21.733413 28626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:26:21.722075 28371 master_runner.cc:186] Error getting master registration for 127.14.142.61:44845: OK, instance_id { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" instance_seqno: 1790457980994906 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 21:26:21.734155 28371 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 21:26:21.738935 28512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } attempt: 5
Sep 26 21:26:21 dist-test-slave-4d5t krb5kdc[28358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457980, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:26:21.757129 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.739929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.14.142.60:40887)
0926 21:26:21.740140 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:21.740145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:21.740159 (+ 14us) server_negotiation.cc:407] Connection header received
0926 21:26:21.740203 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:21.740207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:21.740260 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:21.740377 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:21.741232 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.742019 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.743835 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.744412 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.744612 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.751096 (+ 6484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:21.751101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:21.751105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:21.751139 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:21.754293 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.755024 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.755032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.755036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.755190 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:21.755570 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:21.755576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:21.755579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:21.755898 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:21.756090 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:21.756469 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 21:26:21.756731 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:26:21.757122 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:21.739579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:40887 (local address 127.0.0.1:49606)
0926 21:26:21.739869 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:21.739885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:21.740006 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:21.740401 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:21.740405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:21.740911 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:21.741077 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.741086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.742156 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.742164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.743490 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:21.743503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:21.744631 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:21.744638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:21.744975 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:21.748532 (+ 3557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:21.748607 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:21.750850 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:21.754461 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.754467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.754471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.754895 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.755306 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:21.755312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:21.755316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:21.755454 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:21.756031 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:21.756036 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:21.756193 (+ 157us) client_negotiation.cc:769] Sending connection context
0926 21:26:21.756351 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 21:26:21.756701 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260926 21:26:21.766027 28512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: Bootstrap starting.
I20260926 21:26:21.769394 28512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:21.770563 28512 log.cc:824] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:21.773734 28512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: No bootstrap required, opened a new log
I20260926 21:26:21.782959 28512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.783350 28512 raft_consensus.cc:374] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:21.783473 28512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040a4dc034eb44b682f482f12b8a37fc, State: Initialized, Role: FOLLOWER
I20260926 21:26:21.784148 28512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.785223 28630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:21.785742 28630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:21.786744 28512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: configured and running, proceeding with master startup.
W20260926 21:26:21.797320 28541 tablet.cc:2369] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:21.806255 28445 master_runner.cc:263] No leader master found from master 040a4dc034eb44b682f482f12b8a37fc
I20260926 21:26:21.806470 28445 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260926 21:26:21.811069 28641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:26:21.811230 28641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:26:21.961742 28604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:26:21.962108 28604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.963862 28604 leader_election.cc:288] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4bd1b3b3ec5c4b879b66560a863c2faf (127.14.142.62:44981), 040a4dc034eb44b682f482f12b8a37fc (127.14.142.61:44845)
I20260926 21:26:21.965310 28487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "627e7d4cc542450a874f1939518b1ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040a4dc034eb44b682f482f12b8a37fc" is_pre_election: true
I20260926 21:26:21.965940 28487 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 627e7d4cc542450a874f1939518b1ce5 in term 0.
I20260926 21:26:21.966806 28537 leader_election.cc:302] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 040a4dc034eb44b682f482f12b8a37fc, 627e7d4cc542450a874f1939518b1ce5; no voters:
I20260926 21:26:21.967372 28604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:26:21.967521 28604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:26:21.967618 28604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:21.971915 28413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "627e7d4cc542450a874f1939518b1ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" is_pre_election: true
I20260926 21:26:21.972535 28413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 627e7d4cc542450a874f1939518b1ce5 in term 0.
I20260926 21:26:21.973809 28604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.978112 28413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "627e7d4cc542450a874f1939518b1ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf"
I20260926 21:26:21.978335 28413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:21.981679 28413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 627e7d4cc542450a874f1939518b1ce5 in term 1.
W20260926 21:26:21.981823 28466 tablet.cc:2369] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:21.982252 28537 leader_election.cc:302] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bd1b3b3ec5c4b879b66560a863c2faf, 627e7d4cc542450a874f1939518b1ce5; no voters:
I20260926 21:26:21.985191 28604 leader_election.cc:288] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [CANDIDATE]: Term 1 election: Requested vote from peers 4bd1b3b3ec5c4b879b66560a863c2faf (127.14.142.62:44981), 040a4dc034eb44b682f482f12b8a37fc (127.14.142.61:44845)
I20260926 21:26:21.985988 28604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:21.986446 28604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [term 1 LEADER]: Becoming Leader. State: Replica: 627e7d4cc542450a874f1939518b1ce5, State: Running, Role: LEADER
I20260926 21:26:21.987226 28604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } }
I20260926 21:26:21.998199 28487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "627e7d4cc542450a874f1939518b1ce5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040a4dc034eb44b682f482f12b8a37fc"
I20260926 21:26:21.998499 28487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:22.006647 28487 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 627e7d4cc542450a874f1939518b1ce5 in term 1.
I20260926 21:26:22.029475 28643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 627e7d4cc542450a874f1939518b1ce5. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.030040 28643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:22.041527 28645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:22.044230 28645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:22.063498 28487 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [term 1 FOLLOWER]: Refusing update from remote peer 627e7d4cc542450a874f1939518b1ce5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:22.064692 28643 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:22.072703 28413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [term 1 FOLLOWER]: Refusing update from remote peer 627e7d4cc542450a874f1939518b1ce5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:22.073643 28643 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:22.091615 28630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 627e7d4cc542450a874f1939518b1ce5. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.091953 28630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:22.094295 28646 mvcc.cc:205] Tried to move back new op lower bound from 7333715894518140928 to 7333715894232047616. Current Snapshot: MvccSnapshot[applied={T|T < 7333715894518140928}]
I20260926 21:26:22.096446 28630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.096843 28630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:22.100206 28598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 627e7d4cc542450a874f1939518b1ce5. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.100596 28598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:22.104189 28648 mvcc.cc:205] Tried to move back new op lower bound from 7333715894518140928 to 7333715894232047616. Current Snapshot: MvccSnapshot[applied={T|T < 7333715894518140928}]
I20260926 21:26:22.105948 28643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.106338 28643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:22.109323 28645 catalog_manager.cc:1347] Generated new cluster ID: 160c981b1f6d4f14ae7edd618bb3904f
I20260926 21:26:22.109443 28645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:22.124291 28604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.124869 28604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:22.126017 28598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "627e7d4cc542450a874f1939518b1ce5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bd1b3b3ec5c4b879b66560a863c2faf" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44981 } } peers { permanent_uuid: "040a4dc034eb44b682f482f12b8a37fc" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 44845 } } peers { permanent_uuid: "627e7d4cc542450a874f1939518b1ce5" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 40887 } } }
I20260926 21:26:22.126292 28598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:26:22.141855 28645 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:26:22.143870 28645 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:22.168640 28645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5: Generated new TSK 0
I20260926 21:26:22.169867 28645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:26:22.736322 28626 catalog_manager.cc:1259] Loaded cluster ID: 160c981b1f6d4f14ae7edd618bb3904f
I20260926 21:26:22.736492 28626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: loading cluster ID for follower catalog manager: success
I20260926 21:26:22.739178 28626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: acquiring CA information for follower catalog manager: success
I20260926 21:26:22.741062 28626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4bd1b3b3ec5c4b879b66560a863c2faf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:26:22.813761 28641 catalog_manager.cc:1259] Loaded cluster ID: 160c981b1f6d4f14ae7edd618bb3904f
I20260926 21:26:22.813903 28641 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: loading cluster ID for follower catalog manager: success
I20260926 21:26:22.816372 28641 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: acquiring CA information for follower catalog manager: success
I20260926 21:26:22.818253 28641 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 040a4dc034eb44b682f482f12b8a37fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:26:23.323526 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28371
I20260926 21:26:23.325174 28433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:23.439546 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28371
W20260926 21:26:23.461755 28537 connection.cc:535] client connection to 127.14.142.62:44981 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 21:26:23.461966 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28445
W20260926 21:26:23.462049 28537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260926 21:26:23.463223 28507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:23.463837 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:23.463139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44981 (local address 127.0.0.1:48804)
0926 21:26:23.463585 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:23.463683 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:44981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":244,"threads_started":1}
W20260926 21:26:23.464052 28537 consensus_peers.cc:487] T 00000000000000000000000000000000 P 627e7d4cc542450a874f1939518b1ce5 -> Peer 4bd1b3b3ec5c4b879b66560a863c2faf (127.14.142.62:44981): Couldn't send request to peer 4bd1b3b3ec5c4b879b66560a863c2faf. Status: Network error: Client connection negotiation failed: client connection to 127.14.142.62:44981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 21:26:23.575045 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28445
I20260926 21:26:23.597362 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28520
I20260926 21:26:23.598676 28582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:23.710883 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28520
2026-09-26T21:26:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5171 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:26:23 dist-test-slave-4d5t krb5kdc[28666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:26:23 dist-test-slave-4d5t krb5kdc[28666](info): set up 2 sockets
Sep 26 21:26:23 dist-test-slave-4d5t krb5kdc[28666](info): commencing operation
krb5kdc: starting...
W20260926 21:26:24.869436 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:26:25 dist-test-slave-4d5t krb5kdc[28666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:26:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:26:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:26:25.145435 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:35413
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:35413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:25.520063 28679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:25.520387 28679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:25.520589 28679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:25.528522 28679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:25.528949 28679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:25.529069 28679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:25.529178 28679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:25.529266 28679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:25.541148 28679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:35413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:35413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:25.543316 28679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:25.544991 28679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:25.559612 28685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:25.559512 28684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:25.559563 28687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:25.560781 28679 server_base.cc:1034] running on GCE node
I20260926 21:26:25.561554 28679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:25.563052 28679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:25.564297 28679 hybrid_clock.cc:648] HybridClock initialized: now 1790457985564248 us; error 37 us; skew 500 ppm
I20260926 21:26:25.564751 28679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:25 dist-test-slave-4d5t krb5kdc[28666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457985, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:25.571458 28679 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:25.573405 28679 webserver.cc:460] Webserver started at http://127.14.142.62:39529/ using document root <none> and password file <none>
I20260926 21:26:25.574194 28679 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:25.574337 28679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:25.574692 28679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:25.578019 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "a81a40802f0e435db386d64030a90e11"
format_stamp: "Formatted at 2026-09-26 21:26:25 on dist-test-slave-4d5t"
server_key: "147bf2e2e44e2acdec81f174cfe35713"
server_key_iv: "f2fb91389b107832185142b7dbf43e68"
server_key_version: "encryptionkey@0"
I20260926 21:26:25.578884 28679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "a81a40802f0e435db386d64030a90e11"
format_stamp: "Formatted at 2026-09-26 21:26:25 on dist-test-slave-4d5t"
server_key: "147bf2e2e44e2acdec81f174cfe35713"
server_key_iv: "f2fb91389b107832185142b7dbf43e68"
server_key_version: "encryptionkey@0"
I20260926 21:26:25.584731 28679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260926 21:26:25.588945 28694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:25.590529 28679 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:26:25.590750 28679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "a81a40802f0e435db386d64030a90e11"
format_stamp: "Formatted at 2026-09-26 21:26:25 on dist-test-slave-4d5t"
server_key: "147bf2e2e44e2acdec81f174cfe35713"
server_key_iv: "f2fb91389b107832185142b7dbf43e68"
server_key_version: "encryptionkey@0"
I20260926 21:26:25.590986 28679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:25.615427 28679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:25.620826 28679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:25.621150 28679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:25.642450 28679 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:35413
I20260926 21:26:25.642462 28745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:35413 every 8 connection(s)
I20260926 21:26:25.643885 28679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:25.644908 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28679
I20260926 21:26:25.645154 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:26:25.645829 14904 external_mini_cluster.cc:1428] Setting key 3e51d8c8ce6400e7c6abdb5ee5c97d39
I20260926 21:26:25.648470 28746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:25.662145 28746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11: Bootstrap starting.
Sep 26 21:26:25 dist-test-slave-4d5t krb5kdc[28666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:25.665949 28746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:25.667094 28746 log.cc:824] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:25.670275 28746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11: No bootstrap required, opened a new log
I20260926 21:26:25.674101 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:25.648139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.14.142.62:35413)
0926 21:26:25.649001 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:25.649012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:25.649058 (+ 46us) server_negotiation.cc:407] Connection header received
0926 21:26:25.651264 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:25.651294 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:25.651841 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:25.652316 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:25.655275 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:25.657564 (+ 2289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:25.659097 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:25.659856 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:25.660080 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:25.664944 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:25.664954 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:25.664970 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:25.665011 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:25.668742 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:25.669579 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:25.669585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:25.669592 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:25.669750 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:25.670132 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:25.670138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:25.670141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:25.670737 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:25.671071 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:25.671523 (+ 452us) server_negotiation.cc:299] Negotiation successful
0926 21:26:25.672395 (+ 872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":438,"threads_started":1}
I20260926 21:26:25.679203 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:26:25.679728 28746 raft_consensus.cc:348] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81a40802f0e435db386d64030a90e11" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35413 } }
I20260926 21:26:25.680143 28746 raft_consensus.cc:374] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:25.680255 28746 raft_consensus.cc:729] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a81a40802f0e435db386d64030a90e11, State: Initialized, Role: FOLLOWER
I20260926 21:26:25.680868 28746 consensus_queue.cc:260] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81a40802f0e435db386d64030a90e11" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35413 } }
I20260926 21:26:25.681121 28746 raft_consensus.cc:388] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:25.681288 28746 raft_consensus.cc:482] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:25.681488 28746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:25.684108 28746 raft_consensus.cc:504] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81a40802f0e435db386d64030a90e11" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35413 } }
I20260926 21:26:25.684671 28746 leader_election.cc:302] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a81a40802f0e435db386d64030a90e11; no voters:
I20260926 21:26:25.685334 28746 leader_election.cc:288] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:26:25.685482 28751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:25.686182 28751 raft_consensus.cc:686] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [term 1 LEADER]: Becoming Leader. State: Replica: a81a40802f0e435db386d64030a90e11, State: Running, Role: LEADER
I20260926 21:26:25.686712 28751 consensus_queue.cc:237] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81a40802f0e435db386d64030a90e11" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35413 } }
I20260926 21:26:25.687458 28746 sys_catalog.cc:564] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:25.689826 28753 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a81a40802f0e435db386d64030a90e11. Latest consensus state: current_term: 1 leader_uuid: "a81a40802f0e435db386d64030a90e11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81a40802f0e435db386d64030a90e11" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35413 } } }
I20260926 21:26:25.690524 28753 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:25.690402 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a81a40802f0e435db386d64030a90e11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a81a40802f0e435db386d64030a90e11" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35413 } } }
I20260926 21:26:25.690793 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:25.692680 28758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:25.698498 28758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:25.708212 28758 catalog_manager.cc:1347] Generated new cluster ID: bf0256fbdd174dbc9b5538a84dc7ab89
I20260926 21:26:25.708354 28758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:25.736894 28758 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:26:25.739365 28758 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:25.752507 28758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a81a40802f0e435db386d64030a90e11: Generated new TSK 0
I20260926 21:26:25.753623 28758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:26:25.793864 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28679
I20260926 21:26:25.795380 28741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:25.905489 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28679
2026-09-26T21:26:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2187 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:26:26 dist-test-slave-4d5t krb5kdc[28774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:26:26 dist-test-slave-4d5t krb5kdc[28774](info): set up 2 sockets
Sep 26 21:26:26 dist-test-slave-4d5t krb5kdc[28774](info): commencing operation
krb5kdc: starting...
W20260926 21:26:27.069489 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:26:27 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:26:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:26:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:26:27.345589 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46307
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:27.703461 28787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:27.703884 28787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:27.704137 28787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:27.712385 28787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:27.712535 28787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:27.712690 28787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:27.712805 28787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:27.712891 28787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:27.723141 28787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:27.724545 28787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:27.725936 28787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:27.737798 28795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:27.738162 28792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:27.738013 28793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:27.739508 28787 server_base.cc:1034] running on GCE node
I20260926 21:26:27.740399 28787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:27.742000 28787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:27.743266 28787 hybrid_clock.cc:648] HybridClock initialized: now 1790457987743200 us; error 51 us; skew 500 ppm
I20260926 21:26:27.743765 28787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:27 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457987, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:27.747586 28787 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:27.749955 28787 webserver.cc:460] Webserver started at http://127.14.142.62:46245/ using document root <none> and password file <none>
I20260926 21:26:27.751192 28787 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:27.751392 28787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:27.751770 28787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:27.755029 28787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "afb6f33edcaf4c3b9266dd6eb3145547"
format_stamp: "Formatted at 2026-09-26 21:26:27 on dist-test-slave-4d5t"
server_key: "e60b1f5cb0ab98d2418ff0bef754c9d8"
server_key_iv: "656e346a21870e3c977abaff0c852dd0"
server_key_version: "encryptionkey@0"
I20260926 21:26:27.755851 28787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "afb6f33edcaf4c3b9266dd6eb3145547"
format_stamp: "Formatted at 2026-09-26 21:26:27 on dist-test-slave-4d5t"
server_key: "e60b1f5cb0ab98d2418ff0bef754c9d8"
server_key_iv: "656e346a21870e3c977abaff0c852dd0"
server_key_version: "encryptionkey@0"
I20260926 21:26:27.762681 28787 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 21:26:27.767863 28802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:27.769904 28787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 21:26:27.770176 28787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "afb6f33edcaf4c3b9266dd6eb3145547"
format_stamp: "Formatted at 2026-09-26 21:26:27 on dist-test-slave-4d5t"
server_key: "e60b1f5cb0ab98d2418ff0bef754c9d8"
server_key_iv: "656e346a21870e3c977abaff0c852dd0"
server_key_version: "encryptionkey@0"
I20260926 21:26:27.770428 28787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:27.788797 28787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:27.793387 28787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:27.794024 28787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:27.813824 28787 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:46307
I20260926 21:26:27.813856 28853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:46307 every 8 connection(s)
I20260926 21:26:27.815325 28787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:27.819389 28854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:27.822662 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28787
I20260926 21:26:27.822921 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:26:27.823850 14904 external_mini_cluster.cc:1428] Setting key cc2135769a81b2f86ba5da94dd7ee3f2
I20260926 21:26:27.833868 28854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap starting.
I20260926 21:26:27.838459 28854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:27.839860 28854 log.cc:824] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Log is configured to *not* fsync() on all Append() calls
Sep 26 21:26:27 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:27.844161 28854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: No bootstrap required, opened a new log
I20260926 21:26:27.850534 28857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:27.826375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.14.142.62:46307)
0926 21:26:27.827229 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:27.827242 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:27.827285 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:26:27.829108 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:27.829133 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:27.829558 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:27.829908 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:27.831902 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:27.833351 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:27.835448 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:27.836203 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:27.836348 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:27.841767 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:27.841776 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:27.841793 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:27.841831 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:27.845705 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:27.846606 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:27.846613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:27.846620 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:27.846811 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:27.847264 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:27.847270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:27.847273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:27.847938 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:27.848338 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:27.848841 (+ 503us) server_negotiation.cc:299] Negotiation successful
0926 21:26:27.849341 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":415,"threads_started":1}
I20260926 21:26:27.853585 28854 raft_consensus.cc:348] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:27.854045 28854 raft_consensus.cc:374] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:27.854163 28854 raft_consensus.cc:729] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb6f33edcaf4c3b9266dd6eb3145547, State: Initialized, Role: FOLLOWER
I20260926 21:26:27.854708 28854 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:27.854954 28854 raft_consensus.cc:388] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:27.855119 28854 raft_consensus.cc:482] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:27.855293 28854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:27.858424 28854 raft_consensus.cc:504] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:27.859148 28854 leader_election.cc:302] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afb6f33edcaf4c3b9266dd6eb3145547; no voters:
I20260926 21:26:27.859874 28854 leader_election.cc:288] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:26:27.860021 28859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:27.861256 28859 raft_consensus.cc:686] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 LEADER]: Becoming Leader. State: Replica: afb6f33edcaf4c3b9266dd6eb3145547, State: Running, Role: LEADER
I20260926 21:26:27.861851 28859 consensus_queue.cc:237] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:27.862406 28854 sys_catalog.cc:564] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:27.865010 28861 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afb6f33edcaf4c3b9266dd6eb3145547. Latest consensus state: current_term: 1 leader_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } } }
I20260926 21:26:27.865383 28861 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:27.865401 28860 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } } }
I20260926 21:26:27.865806 28860 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:27.868450 28865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:27.873812 28865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:27.883515 28865 catalog_manager.cc:1347] Generated new cluster ID: f776696d4cc04675810ee4479f4a1a2d
I20260926 21:26:27.883643 28865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:27.907713 28865 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:26:27.909152 28865 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:27.916078 28865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Generated new TSK 0
I20260926 21:26:27.917644 28865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:26:27.986929 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46307
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:28.342795 28880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:28.343138 28880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:28.343310 28880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:28.350871 28880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:28.351028 28880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:28.351292 28880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:26:28.361768 28880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:28.363287 28880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:28.364914 28880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:28.379436 28885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:28.379388 28888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:28.379488 28886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:28.385658 28880 server_base.cc:1034] running on GCE node
I20260926 21:26:28.386647 28880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:28.388134 28880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:28.389438 28880 hybrid_clock.cc:648] HybridClock initialized: now 1790457988389358 us; error 75 us; skew 500 ppm
I20260926 21:26:28.389823 28880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:28 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457988, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:28.393483 28880 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:26:28.395318 28880 webserver.cc:460] Webserver started at http://127.14.142.1:42483/ using document root <none> and password file <none>
I20260926 21:26:28.396107 28880 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:28.396265 28880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:28.396656 28880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:28.399699 28880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "dd105549cbc8c6a743a54315c8971d2e"
server_key_iv: "1c014ecbf6ab729de067f1d9c9e38a25"
server_key_version: "encryptionkey@0"
I20260926 21:26:28.400604 28880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "dd105549cbc8c6a743a54315c8971d2e"
server_key_iv: "1c014ecbf6ab729de067f1d9c9e38a25"
server_key_version: "encryptionkey@0"
I20260926 21:26:28.406175 28880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 21:26:28.410190 28895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:28.411748 28880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 21:26:28.411962 28880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "dd105549cbc8c6a743a54315c8971d2e"
server_key_iv: "1c014ecbf6ab729de067f1d9c9e38a25"
server_key_version: "encryptionkey@0"
I20260926 21:26:28.412212 28880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:28.427660 28880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:28.432108 28880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:28.432432 28880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:28.433468 28880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:28.434873 28880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:28.434981 28880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:28.435137 28880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:28.435215 28880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:28.471894 28880 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:35919
I20260926 21:26:28.471947 29007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:35919 every 8 connection(s)
I20260926 21:26:28.473264 28880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:26:28.475693 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 28880
I20260926 21:26:28.475942 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:26:28.476811 14904 external_mini_cluster.cc:1428] Setting key f73a7f63e1e2ec8d698f693fe2bd3704
Sep 26 21:26:28 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457988, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:28.495689 29010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:28.476019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:60855 (local address 127.14.142.62:46307)
0926 21:26:28.476542 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:28.476571 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:28.477626 (+ 1055us) server_negotiation.cc:407] Connection header received
0926 21:26:28.479481 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:28.479489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:28.479583 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:28.479706 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:28.481754 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:28.482907 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:28.484854 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:28.485579 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:28.485738 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:28.489390 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:28.489395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:28.489398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:28.489436 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:28.492432 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:28.493283 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:28.493293 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:28.493296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:28.493415 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:28.493877 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:28.493886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:28.493888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:28.494163 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:28.494324 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:28.495058 (+ 734us) server_negotiation.cc:299] Negotiation successful
0926 21:26:28.495323 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":271,"threads_started":1}
I20260926 21:26:28.496800 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:28.476640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:60855)
0926 21:26:28.477473 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:28.477530 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:28.479222 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:28.479976 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:28.479986 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:28.480636 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:28.481532 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:28.481555 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:28.483112 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:28.483117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:28.484693 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:28.484708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:28.485829 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:28.485845 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:28.486065 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:28.486849 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:28.486872 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:28.489114 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:28.492665 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:28.492674 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:28.492690 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:28.493141 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:28.493535 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:28.493540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:28.493543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:28.493750 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:28.494297 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:28.494306 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:28.494677 (+ 371us) client_negotiation.cc:769] Sending connection context
0926 21:26:28.494949 (+ 272us) client_negotiation.cc:240] Negotiation successful
0926 21:26:28.495513 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":377,"threads_started":1}
I20260926 21:26:28.498486 29008 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46307
I20260926 21:26:28.498857 29008 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:28.499774 29008 heartbeater.cc:511] Master 127.14.142.62:46307 requested a full tablet report, sending...
I20260926 21:26:28.502074 28819 ts_manager.cc:194] Registered new tserver with Master: 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919)
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:26:28.503715 28819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:60855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:26:28.536680 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46307
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:28.880810 29014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:28.881114 29014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:28.881271 29014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:28.891734 29014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:28.891873 29014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:28.892082 29014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:26:28.903427 29014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:28.904882 29014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:28.906196 29014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:28.919526 29019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:28.919453 29020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:28.919961 29022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:28.920763 29014 server_base.cc:1034] running on GCE node
I20260926 21:26:28.921401 29014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:28.922360 29014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:28.923730 29014 hybrid_clock.cc:648] HybridClock initialized: now 1790457988923659 us; error 57 us; skew 500 ppm
I20260926 21:26:28.924152 29014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:28 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457988, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:28.927685 29014 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:26:28.929545 29014 webserver.cc:460] Webserver started at http://127.14.142.2:37645/ using document root <none> and password file <none>
I20260926 21:26:28.930265 29014 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:28.930411 29014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:28.930727 29014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:28.933780 29014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "b6ecda5b7d7a4b28922c981d065df503"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "8851d99afa9c0ac1670150ca43a041f7"
server_key_iv: "d5cb575e72b0480e377a4383c67c0d5d"
server_key_version: "encryptionkey@0"
I20260926 21:26:28.934674 29014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "b6ecda5b7d7a4b28922c981d065df503"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "8851d99afa9c0ac1670150ca43a041f7"
server_key_iv: "d5cb575e72b0480e377a4383c67c0d5d"
server_key_version: "encryptionkey@0"
I20260926 21:26:28.939935 29014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 21:26:28.944162 29029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:28.945932 29014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:26:28.946110 29014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "b6ecda5b7d7a4b28922c981d065df503"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "8851d99afa9c0ac1670150ca43a041f7"
server_key_iv: "d5cb575e72b0480e377a4383c67c0d5d"
server_key_version: "encryptionkey@0"
I20260926 21:26:28.946295 29014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:28.982019 29014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:28.986716 29014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:28.986979 29014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:28.987979 29014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:28.989531 29014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:28.989622 29014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:28.989722 29014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:28.989766 29014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:29.029371 29014 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:44565
I20260926 21:26:29.029498 29141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:44565 every 8 connection(s)
I20260926 21:26:29.030720 29014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:26:29.032702 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29014
I20260926 21:26:29.032927 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:26:29.033443 14904 external_mini_cluster.cc:1428] Setting key a27bf3b0d0b620eb4d2b7ae0698a6bdd
Sep 26 21:26:29 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457988, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:29.055603 29144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.034142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:36085 (local address 127.14.142.62:46307)
0926 21:26:29.034822 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.034829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.036276 (+ 1447us) server_negotiation.cc:407] Connection header received
0926 21:26:29.037974 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.037979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.038083 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.038209 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:29.039798 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.040540 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.042163 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.042590 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.042730 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.046407 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:29.046416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:29.046423 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:29.046473 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:29.049969 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.051193 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.051201 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.051207 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.051378 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.051976 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.051981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.051987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.052414 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:29.054843 (+ 2429us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.054983 (+ 140us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.055224 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":290,"threads_started":1}
I20260926 21:26:29.055966 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.034934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:36085)
0926 21:26:29.035808 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.035866 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.037694 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.038458 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.038466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.038998 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:29.039624 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.039642 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.040766 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.040770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.042012 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.042024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.042831 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.042839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.043077 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.043799 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:29.043828 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:29.046134 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:29.050198 (+ 4064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:29.050210 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:29.050234 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:29.050958 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:29.051519 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:29.051523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:29.051527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:29.051803 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:29.052615 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:29.052626 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:29.053185 (+ 559us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.053516 (+ 331us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.054190 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":395,"threads_started":1}
I20260926 21:26:29.059005 29142 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46307
I20260926 21:26:29.059449 29142 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:29.060403 29142 heartbeater.cc:511] Master 127.14.142.62:46307 requested a full tablet report, sending...
I20260926 21:26:29.062214 28819 ts_manager.cc:194] Registered new tserver with Master: b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2:44565)
I20260926 21:26:29.063205 28819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:36085
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:26:29.099731 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46307
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:29.471390 29148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:29.471714 29148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:29.471876 29148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:29.479882 29148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:29.480005 29148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:29.480206 29148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:26:29.490270 29148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:29.491693 29148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:29.493131 29148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 21:26:29.507177 29008 heartbeater.cc:503] Master 127.14.142.62:46307 was elected leader, sending a full tablet report...
W20260926 21:26:29.507195 29154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:29.507195 29153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:29.508468 29148 server_base.cc:1034] running on GCE node
W20260926 21:26:29.508765 29156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:29.509593 29148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:29.510759 29148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:29.511986 29148 hybrid_clock.cc:648] HybridClock initialized: now 1790457989511913 us; error 71 us; skew 500 ppm
I20260926 21:26:29.512357 29148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:29 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457989, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:29.515934 29148 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:26:29.517652 29148 webserver.cc:460] Webserver started at http://127.14.142.3:36919/ using document root <none> and password file <none>
I20260926 21:26:29.518383 29148 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:29.518533 29148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:29.518852 29148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:29.521998 29148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "37e86bff957846b280f82ad99567500e"
format_stamp: "Formatted at 2026-09-26 21:26:29 on dist-test-slave-4d5t"
server_key: "892d53fc0ca821547b353d6a7464ffd9"
server_key_iv: "47b3b6650863f63e0bc812fbfa3494c9"
server_key_version: "encryptionkey@0"
I20260926 21:26:29.522812 29148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "37e86bff957846b280f82ad99567500e"
format_stamp: "Formatted at 2026-09-26 21:26:29 on dist-test-slave-4d5t"
server_key: "892d53fc0ca821547b353d6a7464ffd9"
server_key_iv: "47b3b6650863f63e0bc812fbfa3494c9"
server_key_version: "encryptionkey@0"
I20260926 21:26:29.528772 29148 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 21:26:29.533164 29163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:29.534956 29148 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 21:26:29.535214 29148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "37e86bff957846b280f82ad99567500e"
format_stamp: "Formatted at 2026-09-26 21:26:29 on dist-test-slave-4d5t"
server_key: "892d53fc0ca821547b353d6a7464ffd9"
server_key_iv: "47b3b6650863f63e0bc812fbfa3494c9"
server_key_version: "encryptionkey@0"
I20260926 21:26:29.535449 29148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:29.552712 29148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:29.557399 29148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:29.557746 29148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:29.558768 29148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:29.560242 29148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:29.560349 29148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:29.560506 29148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:29.560604 29148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:29.597680 29148 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:44103
I20260926 21:26:29.597770 29275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:44103 every 8 connection(s)
I20260926 21:26:29.599016 29148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:26:29.600092 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29148
I20260926 21:26:29.600288 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:26:29.600916 14904 external_mini_cluster.cc:1428] Setting key a30779d626820b7e511f17405e4ed5f3
Sep 26 21:26:29 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457989, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:29.623031 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.602257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:56373 (local address 127.14.142.62:46307)
0926 21:26:29.602755 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.602760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.603925 (+ 1165us) server_negotiation.cc:407] Connection header received
0926 21:26:29.606128 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.606133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.606271 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.606404 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:29.608081 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.609010 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.611099 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.611530 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.611651 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.616387 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:29.616393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:29.616397 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:29.616444 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:29.619510 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.620357 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.620361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.620363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.620530 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.621095 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.621099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.621101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.621397 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:29.621515 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.622372 (+ 857us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.622660 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":227,"threads_started":1}
I20260926 21:26:29.623749 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.602835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:56373)
0926 21:26:29.603751 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.603803 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.605770 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.606670 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.606679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.607235 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:29.607851 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.607870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.609161 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.609165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.610929 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.610944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.611970 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.611979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.612409 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.613477 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:29.613507 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:29.616091 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:29.619710 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:29.619719 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:29.619736 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:29.620224 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:29.620737 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:29.620740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:29.620743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:29.620967 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:29.621518 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:29.621524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:29.621883 (+ 359us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.622136 (+ 253us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.622571 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":357,"threads_started":1}
I20260926 21:26:29.625547 29276 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46307
I20260926 21:26:29.625886 29276 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:29.626683 29276 heartbeater.cc:511] Master 127.14.142.62:46307 requested a full tablet report, sending...
I20260926 21:26:29.628242 28819 ts_manager.cc:194] Registered new tserver with Master: 37e86bff957846b280f82ad99567500e (127.14.142.3:44103)
I20260926 21:26:29.629283 28819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:56373
I20260926 21:26:29.630319 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:26:29.649454 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.634862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.14.142.62:46307)
0926 21:26:29.635031 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.635035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.635514 (+ 479us) server_negotiation.cc:407] Connection header received
0926 21:26:29.635727 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.635731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.635806 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.635894 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:29.637580 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.638333 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.639915 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.640499 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.640673 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.643179 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:29.643184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:29.643188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:29.643225 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:29.646267 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.647009 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.647013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.647016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.647138 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.647522 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.647526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.647528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.647892 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:29.648079 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.648702 (+ 623us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.649009 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 21:26:29.654659 28819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:26:29.662509 28819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:26:29.698148 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.677210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.14.142.1:35919)
0926 21:26:29.688052 (+ 10842us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.688077 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.689014 (+ 937us) server_negotiation.cc:407] Connection header received
0926 21:26:29.689128 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.689138 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.689393 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.690463 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:29.690717 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.692273 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.695190 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.696815 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.696999 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.697255 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.697637 (+ 382us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.697778 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10632,"thread_start_us":10452,"threads_started":1}
I20260926 21:26:29.698182 29289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.676599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35919 (local address 127.0.0.1:49762)
0926 21:26:29.688485 (+ 11886us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.688541 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.688796 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.689651 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.689659 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.689688 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:29.690197 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.690207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.692417 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.692422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.694962 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.694978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.697092 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.697098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.697348 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.697418 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.697589 (+ 171us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.697835 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11627,"mutex_wait_us":22,"thread_start_us":11516,"threads_started":1}
I20260926 21:26:29.702656 29290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.676701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:44565 (local address 127.0.0.1:47190)
0926 21:26:29.690263 (+ 13562us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.690285 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.690403 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.691403 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.691408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.691436 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:29.691838 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.691850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.695162 (+ 3312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.695171 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.699455 (+ 4284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.699471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.701743 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.701749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.701928 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.701962 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.702100 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.702242 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13390,"mutex_wait_us":4,"thread_start_us":13199,"threads_started":1}
I20260926 21:26:29.702741 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.677094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.14.142.2:44565)
0926 21:26:29.689411 (+ 12317us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.689425 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.690587 (+ 1162us) server_negotiation.cc:407] Connection header received
0926 21:26:29.690708 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.690715 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.690954 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.691157 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:29.693419 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.694994 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.699690 (+ 4696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.701239 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.701389 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.701594 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.702214 (+ 620us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.702396 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12058,"thread_start_us":12324,"threads_started":1}
I20260926 21:26:29.702564 28943 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f (DEFAULT_TABLE table=test-table [id=b9a84bd5a50147f1a567854a562231cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:29.704351 28943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:29.704510 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.690421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:44103 (local address 127.0.0.1:48354)
0926 21:26:29.695669 (+ 5248us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.695691 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.695827 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.697243 (+ 1416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.697247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.697274 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:29.697758 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.697771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.699669 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.699676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.701550 (+ 1874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.701582 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.703689 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.703694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.703811 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.703833 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.703944 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.704083 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5091,"mutex_wait_us":15,"thread_start_us":4925,"threads_started":1}
I20260926 21:26:29.704509 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.690726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.14.142.3:44103)
0926 21:26:29.694665 (+ 3939us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.694671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.695957 (+ 1286us) server_negotiation.cc:407] Connection header received
0926 21:26:29.696062 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.696085 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.696764 (+ 679us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.697008 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:29.697956 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.699554 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.701759 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.703213 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.703341 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.703576 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.703987 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.704172 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3776,"thread_start_us":3598,"threads_started":1}
I20260926 21:26:29.706931 29077 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f (DEFAULT_TABLE table=test-table [id=b9a84bd5a50147f1a567854a562231cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:29.707693 29211 tablet_service.cc:1467] Processing CreateTablet for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f (DEFAULT_TABLE table=test-table [id=b9a84bd5a50147f1a567854a562231cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:29.708323 29077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:29.708809 29211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:29.721236 29296 tablet_bootstrap.cc:492] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Bootstrap starting.
I20260926 21:26:29.721850 29295 tablet_bootstrap.cc:492] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Bootstrap starting.
I20260926 21:26:29.725291 29297 tablet_bootstrap.cc:492] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Bootstrap starting.
I20260926 21:26:29.725880 29295 tablet_bootstrap.cc:654] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:29.725939 29296 tablet_bootstrap.cc:654] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:29.727330 29296 log.cc:824] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:29.727330 29295 log.cc:824] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:29.728634 29297 tablet_bootstrap.cc:654] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:29.729892 29297 log.cc:824] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:29.733300 29297 tablet_bootstrap.cc:492] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: No bootstrap required, opened a new log
I20260926 21:26:29.733336 29295 tablet_bootstrap.cc:492] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: No bootstrap required, opened a new log
I20260926 21:26:29.733376 29296 tablet_bootstrap.cc:492] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: No bootstrap required, opened a new log
I20260926 21:26:29.733613 29297 ts_tablet_manager.cc:1397] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 21:26:29.733623 29295 ts_tablet_manager.cc:1397] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260926 21:26:29.733654 29296 ts_tablet_manager.cc:1397] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260926 21:26:29.742503 29297 raft_consensus.cc:348] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.742991 29297 raft_consensus.cc:374] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:29.743114 29297 raft_consensus.cc:729] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ecda5b7d7a4b28922c981d065df503, State: Initialized, Role: FOLLOWER
I20260926 21:26:29.743861 29297 consensus_queue.cc:260] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.744817 29295 raft_consensus.cc:348] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.745152 29142 heartbeater.cc:503] Master 127.14.142.62:46307 was elected leader, sending a full tablet report...
I20260926 21:26:29.745337 29295 raft_consensus.cc:374] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:29.745448 29295 raft_consensus.cc:729] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e86bff957846b280f82ad99567500e, State: Initialized, Role: FOLLOWER
I20260926 21:26:29.745271 29296 raft_consensus.cc:348] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.745715 29296 raft_consensus.cc:374] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:29.745898 29297 ts_tablet_manager.cc:1428] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260926 21:26:29.745959 29296 raft_consensus.cc:729] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f82abb48a9f4f0e8c8fb39aa2231aff, State: Initialized, Role: FOLLOWER
I20260926 21:26:29.746129 29295 consensus_queue.cc:260] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.746671 29296 consensus_queue.cc:260] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.747344 29276 heartbeater.cc:503] Master 127.14.142.62:46307 was elected leader, sending a full tablet report...
I20260926 21:26:29.748632 29296 ts_tablet_manager.cc:1428] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260926 21:26:29.750464 29295 ts_tablet_manager.cc:1428] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
W20260926 21:26:29.783062 29143 tablet.cc:2369] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:29.836858 29301 raft_consensus.cc:482] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:26:29.837353 29301 raft_consensus.cc:504] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.839815 29301 leader_election.cc:288] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919), 37e86bff957846b280f82ad99567500e (127.14.142.3:44103)
I20260926 21:26:29.848371 29305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.840208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:44103 (local address 127.14.142.2:36837)
0926 21:26:29.841259 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.841286 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.841475 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.841961 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.841967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.842014 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:29.842663 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.842673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.844141 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.844153 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.846083 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.846099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.847414 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.847420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.847699 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.847738 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.847864 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.847990 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":108,"spinlock_wait_cycles":5888,"thread_start_us":911,"threads_started":1}
I20260926 21:26:29.848636 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.840480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:36837 (local address 127.14.142.3:44103)
0926 21:26:29.840726 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.840734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.841411 (+ 677us) server_negotiation.cc:407] Connection header received
0926 21:26:29.841666 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.841671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.841757 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.841875 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:29.842833 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.843977 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.846300 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.847188 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.847329 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.847388 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.848014 (+ 626us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.848235 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 21:26:29.849211 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.840620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:43189 (local address 127.14.142.1:35919)
0926 21:26:29.840913 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.840921 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.841590 (+ 669us) server_negotiation.cc:407] Connection header received
0926 21:26:29.841723 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.841728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.841804 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.841893 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:29.842833 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.843985 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.846389 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.847444 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.847713 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.847788 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.848846 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.849025 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260926 21:26:29.849476 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.840238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35919 (local address 127.14.142.2:43189)
0926 21:26:29.841464 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:29.841480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:29.841594 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:29.841961 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:29.841967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:29.842009 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:29.842594 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.842615 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.844221 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.844225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.846211 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:29.846229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.847941 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.847951 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:29.848236 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.848280 (+ 44us) client_negotiation.cc:769] Sending connection context
0926 21:26:29.848458 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 21:26:29.849013 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":69,"spinlock_wait_cycles":768,"thread_start_us":611,"threads_started":1}
I20260926 21:26:29.849920 29231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd780fe3f7b4d4baf3dc9449ad4e50f" candidate_uuid: "b6ecda5b7d7a4b28922c981d065df503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e86bff957846b280f82ad99567500e" is_pre_election: true
I20260926 21:26:29.850538 29231 raft_consensus.cc:2454] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ecda5b7d7a4b28922c981d065df503 in term 0.
I20260926 21:26:29.850386 28963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd780fe3f7b4d4baf3dc9449ad4e50f" candidate_uuid: "b6ecda5b7d7a4b28922c981d065df503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" is_pre_election: true
W20260926 21:26:29.850729 29277 tablet.cc:2369] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:29.850951 28963 raft_consensus.cc:2454] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6ecda5b7d7a4b28922c981d065df503 in term 0.
I20260926 21:26:29.851290 29033 leader_election.cc:302] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e86bff957846b280f82ad99567500e, b6ecda5b7d7a4b28922c981d065df503; no voters:
I20260926 21:26:29.851742 29301 raft_consensus.cc:2781] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:26:29.851883 29301 raft_consensus.cc:482] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:26:29.851977 29301 raft_consensus.cc:3037] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:29.854240 29301 raft_consensus.cc:504] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.854878 29301 leader_election.cc:288] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [CANDIDATE]: Term 1 election: Requested vote from peers 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919), 37e86bff957846b280f82ad99567500e (127.14.142.3:44103)
I20260926 21:26:29.855413 29231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd780fe3f7b4d4baf3dc9449ad4e50f" candidate_uuid: "b6ecda5b7d7a4b28922c981d065df503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e86bff957846b280f82ad99567500e"
I20260926 21:26:29.855479 28963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fd780fe3f7b4d4baf3dc9449ad4e50f" candidate_uuid: "b6ecda5b7d7a4b28922c981d065df503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
I20260926 21:26:29.855599 29231 raft_consensus.cc:3037] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:29.855746 28963 raft_consensus.cc:3037] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:29.858021 29231 raft_consensus.cc:2454] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ecda5b7d7a4b28922c981d065df503 in term 1.
I20260926 21:26:29.858085 28963 raft_consensus.cc:2454] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6ecda5b7d7a4b28922c981d065df503 in term 1.
I20260926 21:26:29.858515 29033 leader_election.cc:302] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e86bff957846b280f82ad99567500e, b6ecda5b7d7a4b28922c981d065df503; no voters:
I20260926 21:26:29.858889 29301 raft_consensus.cc:2781] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:29.859373 29301 raft_consensus.cc:686] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 1 LEADER]: Becoming Leader. State: Replica: b6ecda5b7d7a4b28922c981d065df503, State: Running, Role: LEADER
I20260926 21:26:29.859897 29301 consensus_queue.cc:237] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } }
I20260926 21:26:29.865940 28818 catalog_manager.cc:5495] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2). New cstate: current_term: 1 leader_uuid: "b6ecda5b7d7a4b28922c981d065df503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } health_report { overall_health: UNKNOWN } } }
I20260926 21:26:29.898773 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.891401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.14.142.2:44565)
0926 21:26:29.891811 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.891818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.891841 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:26:29.891994 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.891998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.892126 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.892315 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:26:29.892814 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.894054 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.896240 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.896760 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.896877 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.897481 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:26:29.898056 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:26:29.898184 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.898328 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.898407 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260926 21:26:29.907967 28963 raft_consensus.cc:1261] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Refusing update from remote peer b6ecda5b7d7a4b28922c981d065df503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:29.908023 29231 raft_consensus.cc:1261] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Refusing update from remote peer b6ecda5b7d7a4b28922c981d065df503: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:29.909180 29306 consensus_queue.cc:1035] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:29.909612 29301 consensus_queue.cc:1035] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:29.922214 29309 mvcc.cc:205] Tried to move back new op lower bound from 7333715926651326464 to 7333715926470995968. Current Snapshot: MvccSnapshot[applied={T|T < 7333715926651326464}]
I20260926 21:26:29.945091 28818 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:29.945392 28818 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:29.949381 28818 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Sending DeleteTablet for 3 replicas of tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f
I20260926 21:26:29.950839 29077 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:29 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:47190
I20260926 21:26:29.950811 28943 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:29 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:49762
I20260926 21:26:29.950870 29211 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:29 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:48354
I20260926 21:26:29.952158 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28880
I20260926 21:26:29.962116 29321 tablet_replica.cc:330] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: stopping tablet replica
I20260926 21:26:29.964015 29319 tablet_replica.cc:330] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: stopping tablet replica
I20260926 21:26:29.965070 29319 raft_consensus.cc:2229] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:29.965582 29321 raft_consensus.cc:2229] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:26:29.966578 29321 raft_consensus.cc:2258] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:29.967317 29319 raft_consensus.cc:2258] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:29.971244 29319 ts_tablet_manager.cc:1905] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:29.972165 29321 ts_tablet_manager.cc:1905] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:29.976191 29320 tablet_replica.cc:330] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: stopping tablet replica
Sep 26 21:26:29 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:26:29.982179 29321 ts_tablet_manager.cc:1918] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:29.982419 29321 log.cc:1196] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/4fd780fe3f7b4d4baf3dc9449ad4e50f
I20260926 21:26:29.982908 29319 ts_tablet_manager.cc:1918] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:29.983037 29321 ts_tablet_manager.cc:1939] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P b6ecda5b7d7a4b28922c981d065df503: Deleting consensus metadata
I20260926 21:26:29.983517 29320 raft_consensus.cc:2229] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:29.984187 29320 raft_consensus.cc:2258] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:29.985011 29319 log.cc:1196] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/4fd780fe3f7b4d4baf3dc9449ad4e50f
I20260926 21:26:29.985630 29319 ts_tablet_manager.cc:1939] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 6f82abb48a9f4f0e8c8fb39aa2231aff: Deleting consensus metadata
I20260926 21:26:29.986665 28806 catalog_manager.cc:4841] TS b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2:44565): tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f (table test-table [id=b9a84bd5a50147f1a567854a562231cc]) successfully deleted
I20260926 21:26:29.986761 29320 ts_tablet_manager.cc:1905] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:29.987532 28804 catalog_manager.cc:4841] TS 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919): tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f (table test-table [id=b9a84bd5a50147f1a567854a562231cc]) successfully deleted
I20260926 21:26:29.990681 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:29.953865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.14.142.1:35919)
0926 21:26:29.954110 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:29.954119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:29.967647 (+ 13528us) server_negotiation.cc:407] Connection header received
0926 21:26:29.968410 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:29.968416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:29.968508 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:29.968660 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:29.971281 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.972375 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.974570 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:29.975258 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:29.975378 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:29.981873 (+ 6495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:29.981890 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:29.981910 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:29.981962 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:29.986163 (+ 4201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.987898 (+ 1735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.987907 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.987911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.988082 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:29.988580 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:29.988586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:29.988589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:29.989060 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:29.989355 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:29.989987 (+ 632us) server_negotiation.cc:299] Negotiation successful
0926 21:26:29.990233 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:26:29.992264 29003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:29.994925 29320 ts_tablet_manager.cc:1918] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:29.995111 29320 log.cc:1196] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/4fd780fe3f7b4d4baf3dc9449ad4e50f
I20260926 21:26:29.995637 29320 ts_tablet_manager.cc:1939] T 4fd780fe3f7b4d4baf3dc9449ad4e50f P 37e86bff957846b280f82ad99567500e: Deleting consensus metadata
I20260926 21:26:29.997678 28806 catalog_manager.cc:4841] TS 37e86bff957846b280f82ad99567500e (127.14.142.3:44103): tablet 4fd780fe3f7b4d4baf3dc9449ad4e50f (table test-table [id=b9a84bd5a50147f1a567854a562231cc]) successfully deleted
I20260926 21:26:30.102267 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28880
I20260926 21:26:30.123008 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29014
Sep 26 21:26:30 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:26:30.148650 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:30.123950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.14.142.2:44565)
0926 21:26:30.124212 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:30.124218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:30.124245 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:26:30.124390 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:30.124397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:30.124501 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:30.124657 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:30.126008 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:30.127243 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:30.129420 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:30.130146 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:30.130305 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:30.140210 (+ 9905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:30.140227 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:30.140253 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:30.140316 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:30.144498 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:30.145815 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:30.145827 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:30.145830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:30.146050 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:30.146754 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:30.146759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:30.146762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:30.147278 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:30.147645 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:30.147931 (+ 286us) server_negotiation.cc:299] Negotiation successful
0926 21:26:30.148216 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:26:30.149891 29137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:30.273098 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29014
I20260926 21:26:30.293440 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29148
Sep 26 21:26:30 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:26:30.312093 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:30.294428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.14.142.3:44103)
0926 21:26:30.294664 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:30.294669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:30.294692 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:26:30.294865 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:30.294870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:30.294954 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:30.295050 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:30.296341 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:30.297408 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:30.299000 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:30.299451 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:30.299613 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:30.305105 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:30.305124 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:30.305141 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:30.305193 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:30.308171 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:30.309467 (+ 1296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:30.309479 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:30.309481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:30.309720 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:30.310216 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:30.310224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:30.310227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:30.310723 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:30.311019 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:30.311397 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 21:26:30.311620 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:26:30.312948 29271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:30.407214 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29148
I20260926 21:26:30.425879 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 28787
I20260926 21:26:30.427316 28849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:30.542860 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 28787
I20260926 21:26:30.566391 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46307
--webserver_interface=127.14.142.62
--webserver_port=46245
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:30.933071 29327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:30.933426 29327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:30.933580 29327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:30.941242 29327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:30.941371 29327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:30.941443 29327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:30.941538 29327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:30.941618 29327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:30.952080 29327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=46245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:30.953560 29327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:30.954968 29327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:30.981740 29333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:30.981761 29336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:30.981761 29334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:30.982774 29327 server_base.cc:1034] running on GCE node
I20260926 21:26:30.983755 29327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:30.985646 29327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:30.987074 29327 hybrid_clock.cc:648] HybridClock initialized: now 1790457990986973 us; error 99 us; skew 500 ppm
I20260926 21:26:30.987588 29327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:30 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457990, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:30.996140 29327 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:30.998442 29327 webserver.cc:460] Webserver started at http://127.14.142.62:46245/ using document root <none> and password file <none>
I20260926 21:26:30.999343 29327 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:30.999526 29327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:31.008448 29327 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 21:26:31.012291 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:31.014137 29327 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:26:31.014465 29327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "afb6f33edcaf4c3b9266dd6eb3145547"
format_stamp: "Formatted at 2026-09-26 21:26:27 on dist-test-slave-4d5t"
server_key: "e60b1f5cb0ab98d2418ff0bef754c9d8"
server_key_iv: "656e346a21870e3c977abaff0c852dd0"
server_key_version: "encryptionkey@0"
I20260926 21:26:31.015705 29327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:31.042959 29327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:31.049266 29327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:31.049619 29327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:31.078454 29327 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:46307
I20260926 21:26:31.078490 29395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:46307 every 8 connection(s)
I20260926 21:26:31.080116 29327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:31.086779 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29327
I20260926 21:26:31.087837 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:35919
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:46307
--webserver_port=42483
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:31.088254 29396 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:26:31.094086 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap starting.
I20260926 21:26:31.116182 29396 log.cc:824] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:31.130005 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:26:31.130612 29396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap complete.
I20260926 21:26:31.140730 29396 raft_consensus.cc:348] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:31.141674 29396 raft_consensus.cc:729] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb6f33edcaf4c3b9266dd6eb3145547, State: Initialized, Role: FOLLOWER
I20260926 21:26:31.142441 29396 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:31.142763 29396 raft_consensus.cc:388] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:31.142895 29396 raft_consensus.cc:482] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:31.143080 29396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 1 FOLLOWER]: Advancing to term 2
I20260926 21:26:31.149173 29396 raft_consensus.cc:504] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:31.149850 29396 leader_election.cc:302] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afb6f33edcaf4c3b9266dd6eb3145547; no voters:
I20260926 21:26:31.150581 29396 leader_election.cc:288] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 21:26:31.150743 29400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 21:26:31.152025 29400 raft_consensus.cc:686] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 LEADER]: Becoming Leader. State: Replica: afb6f33edcaf4c3b9266dd6eb3145547, State: Running, Role: LEADER
I20260926 21:26:31.152712 29400 consensus_queue.cc:237] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:31.153075 29396 sys_catalog.cc:564] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:31.155464 29402 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afb6f33edcaf4c3b9266dd6eb3145547. Latest consensus state: current_term: 2 leader_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } } }
I20260926 21:26:31.155432 29401 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } } }
I20260926 21:26:31.156018 29402 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:31.156023 29401 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:31.160916 29407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:31.170910 29407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:31.172987 29407 catalog_manager.cc:1259] Loaded cluster ID: f776696d4cc04675810ee4479f4a1a2d
I20260926 21:26:31.173197 29407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:31.177387 29407 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 21:26:31.192274 29407 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 21:26:31.200531 29407 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:31.202124 29407 catalog_manager.cc:5879] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Loaded TSK: 0
I20260926 21:26:31.203329 29407 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 21:26:31.505880 29398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:31.506242 29398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:31.506414 29398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:31.515115 29398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:31.515305 29398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:31.515504 29398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:26:31.527300 29398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:35919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=42483
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:31.528841 29398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:31.530201 29398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:31.546670 29424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:31.546983 29423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:31.547320 29426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:31.548187 29398 server_base.cc:1034] running on GCE node
I20260926 21:26:31.549386 29398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:31.550861 29398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:31.552178 29398 hybrid_clock.cc:648] HybridClock initialized: now 1790457991552107 us; error 57 us; skew 500 ppm
I20260926 21:26:31.552708 29398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:31 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457991, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:31.556433 29398 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:26:31.558470 29398 webserver.cc:460] Webserver started at http://127.14.142.1:42483/ using document root <none> and password file <none>
I20260926 21:26:31.559217 29398 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:31.559383 29398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:31.566440 29398 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 21:26:31.569674 29433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:31.571398 29398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:26:31.571612 29398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "dd105549cbc8c6a743a54315c8971d2e"
server_key_iv: "1c014ecbf6ab729de067f1d9c9e38a25"
server_key_version: "encryptionkey@0"
I20260926 21:26:31.572764 29398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:31.588738 29398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:31.593478 29398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:31.593801 29398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:31.595041 29398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:31.596665 29398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:31.596786 29398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:31.596946 29398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:31.597033 29398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:31.634724 29398 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:35919
I20260926 21:26:31.634821 29547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:35919 every 8 connection(s)
I20260926 21:26:31.636278 29398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:26:31.641492 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29398
I20260926 21:26:31.642693 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:44565
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:46307
--webserver_port=37645
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:26:31 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457991, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:31.669241 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:31.640611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:57231)
0926 21:26:31.641850 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:31.642037 (+ 187us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:31.645578 (+ 3541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:31.647518 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:31.647546 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:31.648442 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:31.649300 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:31.649331 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:31.651091 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:31.651099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:31.652444 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:31.652463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:31.654078 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:31.654084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:31.654509 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:31.655347 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:31.655371 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:31.658642 (+ 3271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:31.664002 (+ 5360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:31.664011 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:31.664025 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:31.664642 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:31.665342 (+ 700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:31.665348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:31.665351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:31.665792 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:31.667012 (+ 1220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:31.667020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:31.667374 (+ 354us) client_negotiation.cc:769] Sending connection context
0926 21:26:31.667641 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 21:26:31.668083 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":440,"threads_started":1}
I20260926 21:26:31.669804 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:31.640269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:57231 (local address 127.14.142.62:46307)
0926 21:26:31.641021 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:31.641039 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:31.642323 (+ 1284us) server_negotiation.cc:407] Connection header received
0926 21:26:31.646441 (+ 4118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:31.646457 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:31.646907 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:31.647262 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:31.649628 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:31.650887 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:31.652645 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:31.653826 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:31.654035 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:31.659026 (+ 4991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:31.659041 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:31.659066 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:31.659139 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:31.663786 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:31.664826 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:31.664839 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:31.664850 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:31.665098 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:31.665993 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:31.665999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:31.666005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:31.666811 (+ 806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:31.667080 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:31.667956 (+ 876us) server_negotiation.cc:299] Negotiation successful
0926 21:26:31.668458 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":307,"threads_started":1}
I20260926 21:26:31.673195 29548 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46307
I20260926 21:26:31.673563 29548 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:31.674484 29548 heartbeater.cc:511] Master 127.14.142.62:46307 requested a full tablet report, sending...
I20260926 21:26:31.677109 29360 ts_manager.cc:194] Registered new tserver with Master: 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919)
I20260926 21:26:31.678592 29360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:57231
W20260926 21:26:32.008956 29552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:32.009316 29552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:32.009498 29552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:32.017015 29552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:32.017186 29552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:32.017381 29552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:26:32.027740 29552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:44565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=37645
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:32.029429 29552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:32.030886 29552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:32.045918 29557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:32.046120 29558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:32.046366 29560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:32.047550 29552 server_base.cc:1034] running on GCE node
I20260926 21:26:32.048218 29552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:32.049330 29552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:32.050565 29552 hybrid_clock.cc:648] HybridClock initialized: now 1790457992050522 us; error 31 us; skew 500 ppm
I20260926 21:26:32.050998 29552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:32 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457992, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:32.054937 29552 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:26:32.056861 29552 webserver.cc:460] Webserver started at http://127.14.142.2:37645/ using document root <none> and password file <none>
I20260926 21:26:32.057736 29552 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:32.057865 29552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:32.064148 29552 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260926 21:26:32.067561 29567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:32.069401 29552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:26:32.069620 29552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "b6ecda5b7d7a4b28922c981d065df503"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "8851d99afa9c0ac1670150ca43a041f7"
server_key_iv: "d5cb575e72b0480e377a4383c67c0d5d"
server_key_version: "encryptionkey@0"
I20260926 21:26:32.070791 29552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:32.102710 29552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:32.108327 29552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:32.108716 29552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:32.110404 29552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:32.112272 29552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:32.112391 29552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:32.112586 29552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:32.112696 29552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260926 21:26:32.151731 29552 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:44565
I20260926 21:26:32.151789 29679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:44565 every 8 connection(s)
I20260926 21:26:32.153398 29552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:26:32.163671 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29552
I20260926 21:26:32.164880 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:44103
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:46307
--webserver_port=36919
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:26:32 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457992, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:32.177747 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.156231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:47893 (local address 127.14.142.62:46307)
0926 21:26:32.156458 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.156463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.157718 (+ 1255us) server_negotiation.cc:407] Connection header received
0926 21:26:32.159643 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.159649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.159857 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.160012 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:32.161696 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.162384 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.165108 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.165617 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.165950 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.170135 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:32.170143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:32.170147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:32.170199 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:32.173730 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:32.174631 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:32.174635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:32.174637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:32.174765 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:32.175398 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:32.175407 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:32.175416 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:32.175765 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:32.175957 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.176997 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.177285 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260926 21:26:32.178926 29683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.156726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:47893)
0926 21:26:32.157475 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.157568 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.159333 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.160277 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.160286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.160890 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:32.161481 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.161500 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.162552 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.162561 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.164844 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.164865 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.166006 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.166017 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.166404 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.167464 (+ 1060us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:32.167491 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:32.169812 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:32.173957 (+ 4145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:32.173968 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:32.173992 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:32.174458 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:32.174898 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:32.174904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:32.174907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:32.175230 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:32.175958 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:32.175969 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:32.176464 (+ 495us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.176848 (+ 384us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.177510 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":305,"threads_started":1}
I20260926 21:26:32.180621 29680 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46307
I20260926 21:26:32.180979 29680 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:32.181783 29680 heartbeater.cc:511] Master 127.14.142.62:46307 requested a full tablet report, sending...
I20260926 21:26:32.183351 29360 ts_manager.cc:194] Registered new tserver with Master: b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2:44565)
I20260926 21:26:32.184298 29360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:47893
W20260926 21:26:32.530222 29684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:32.530618 29684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:32.530807 29684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:32.539824 29684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:32.539995 29684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:32.540356 29684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:26:32.553930 29684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:44103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=36919
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:32.555473 29684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:32.557014 29684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:32.572606 29692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:32.573143 29689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:32.573196 29690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:32.574694 29684 server_base.cc:1034] running on GCE node
I20260926 21:26:32.575469 29684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:32.576710 29684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:32.577963 29684 hybrid_clock.cc:648] HybridClock initialized: now 1790457992577890 us; error 72 us; skew 500 ppm
I20260926 21:26:32.578357 29684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:32 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457992, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:32.582212 29684 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:26:32.584137 29684 webserver.cc:460] Webserver started at http://127.14.142.3:36919/ using document root <none> and password file <none>
I20260926 21:26:32.585022 29684 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:32.585212 29684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:32.591567 29684 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260926 21:26:32.594944 29699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:32.596796 29684 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 21:26:32.597000 29684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "37e86bff957846b280f82ad99567500e"
format_stamp: "Formatted at 2026-09-26 21:26:29 on dist-test-slave-4d5t"
server_key: "892d53fc0ca821547b353d6a7464ffd9"
server_key_iv: "47b3b6650863f63e0bc812fbfa3494c9"
server_key_version: "encryptionkey@0"
I20260926 21:26:32.598066 29684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:32.615234 29684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:32.621300 29684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:32.621650 29684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:32.622730 29684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:32.624202 29684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:32.624328 29684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:32.624467 29684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:32.624555 29684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:32.667426 29684 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:44103
I20260926 21:26:32.667549 29811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:44103 every 8 connection(s)
I20260926 21:26:32.669162 29684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:26:32.675204 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29684
I20260926 21:26:32.682989 29548 heartbeater.cc:503] Master 127.14.142.62:46307 was elected leader, sending a full tablet report...
Sep 26 21:26:32 dist-test-slave-4d5t krb5kdc[28774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457992, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:32.693218 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.676668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.14.142.62:46307)
0926 21:26:32.677631 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.677637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.677657 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:26:32.677731 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.677736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.677811 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.677927 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:32.679951 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.680774 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.682273 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.682979 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.683129 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.686580 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:32.686586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:32.686591 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:32.686633 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:32.690122 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:32.691058 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:32.691065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:32.691068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:32.691248 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:32.691652 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:32.691657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:32.691660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:32.692011 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:32.692315 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.692668 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.692849 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":848,"thread_start_us":687,"threads_started":1}
I20260926 21:26:32.694926 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.672019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:41539 (local address 127.14.142.62:46307)
0926 21:26:32.672232 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.672236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.673409 (+ 1173us) server_negotiation.cc:407] Connection header received
0926 21:26:32.675382 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.675388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.675484 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.675601 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:32.677945 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.679348 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.681834 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.682468 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.682622 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.687316 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:32.687321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:32.687325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:32.687367 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:32.690784 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:32.691841 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:32.691845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:32.691847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:32.691973 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:32.692522 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:32.692527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:32.692532 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:32.692921 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:32.693065 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.694208 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.694577 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":80}
I20260926 21:26:32.695892 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.672409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:41539)
0926 21:26:32.673224 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.673268 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.675093 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.675857 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.675866 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.676686 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:32.677706 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.677732 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.679805 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.679816 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.681611 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.681647 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.682768 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.682777 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.683087 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.683976 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:32.684014 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:32.686990 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:32.691017 (+ 4027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:32.691029 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:32.691051 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:32.691684 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:32.692097 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:32.692104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:32.692109 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:32.692365 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:32.693070 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:32.693078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:32.693599 (+ 521us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.693911 (+ 312us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.694433 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":374,"threads_started":1}
I20260926 21:26:32.697576 29812 heartbeater.cc:347] Connected to a master server at 127.14.142.62:46307
I20260926 21:26:32.697930 29812 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:32.698704 29812 heartbeater.cc:511] Master 127.14.142.62:46307 requested a full tablet report, sending...
I20260926 21:26:32.700379 29360 ts_manager.cc:194] Registered new tserver with Master: 37e86bff957846b280f82ad99567500e (127.14.142.3:44103)
I20260926 21:26:32.701385 29360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:41539
I20260926 21:26:32.710300 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:26:32.718057 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.711360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43328 (local address 127.14.142.62:46307)
0926 21:26:32.711594 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.711604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.712052 (+ 448us) server_negotiation.cc:407] Connection header received
0926 21:26:32.712233 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.712237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.712380 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.712524 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:26:32.713077 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.713932 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.715595 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.716089 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.716196 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.716793 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:26:32.717345 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:26:32.717468 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.717649 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.717730 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:26:32.719127 29360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:26:32.727619 29360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:26:32.765668 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.742098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:44103 (local address 127.0.0.1:48372)
0926 21:26:32.755094 (+ 12996us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.755142 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.755326 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.756205 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.756217 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.756261 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:32.756810 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.756834 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.759801 (+ 2967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.759810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.762899 (+ 3089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.762914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.764713 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.764719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.764912 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.764960 (+ 48us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.765101 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.765294 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12727,"thread_start_us":12453,"threads_started":1}
I20260926 21:26:32.765726 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.742484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.14.142.3:44103)
0926 21:26:32.754596 (+ 12112us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.754611 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.755538 (+ 927us) server_negotiation.cc:407] Connection header received
0926 21:26:32.755654 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.755664 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.755941 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.757107 (+ 1166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:32.757438 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.759581 (+ 2143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.763074 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.764472 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.764631 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.764893 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.765205 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.765373 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11845,"thread_start_us":11629,"threads_started":1}
I20260926 21:26:32.770298 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.742447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35919 (local address 127.0.0.1:49782)
0926 21:26:32.759036 (+ 16589us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.759056 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.759256 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.761808 (+ 2552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.761815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.761852 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:32.762322 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.762335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.764244 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.764249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.767562 (+ 3313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.767576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.769339 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.769345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.769616 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.769662 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.769776 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.769913 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16448,"thread_start_us":16309,"threads_started":1}
I20260926 21:26:32.770315 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.743172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.14.142.1:35919)
0926 21:26:32.758331 (+ 15159us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.758344 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.759147 (+ 803us) server_negotiation.cc:407] Connection header received
0926 21:26:32.759478 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.759490 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.759798 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.760002 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:32.762575 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.764074 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.767735 (+ 3661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.769126 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.769281 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.769618 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.769832 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.770001 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14911,"thread_start_us":15157,"threads_started":1}
I20260926 21:26:32.769793 29747 tablet_service.cc:1467] Processing CreateTablet for tablet 8932db7da88f45f381fccd4bca66bca6 (DEFAULT_TABLE table=test-table [id=a5ce987524fb4539a962c18eff1625f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:32.771355 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.759124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:44565 (local address 127.0.0.1:47222)
0926 21:26:32.762907 (+ 3783us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.762924 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.763036 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.763623 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.763628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.763654 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:32.764014 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.764025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.765879 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.765886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.768212 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.768228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.770381 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.770388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.770763 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.770797 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.770903 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.771052 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3647,"thread_start_us":3504,"threads_started":1}
I20260926 21:26:32.771382 29747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8932db7da88f45f381fccd4bca66bca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:32.771485 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.755848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.14.142.2:44565)
0926 21:26:32.762954 (+ 7106us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.762962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.762992 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:26:32.763177 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.763185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.763398 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.763612 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:32.764225 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.765698 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.768408 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.769875 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.770019 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.770198 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.770987 (+ 789us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.771159 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6459,"thread_start_us":6301,"threads_started":1}
I20260926 21:26:32.774318 29482 tablet_service.cc:1467] Processing CreateTablet for tablet 8932db7da88f45f381fccd4bca66bca6 (DEFAULT_TABLE table=test-table [id=a5ce987524fb4539a962c18eff1625f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:32.775064 29615 tablet_service.cc:1467] Processing CreateTablet for tablet 8932db7da88f45f381fccd4bca66bca6 (DEFAULT_TABLE table=test-table [id=a5ce987524fb4539a962c18eff1625f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:32.775841 29482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8932db7da88f45f381fccd4bca66bca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:32.776295 29615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8932db7da88f45f381fccd4bca66bca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:32.786988 29829 tablet_bootstrap.cc:492] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Bootstrap starting.
I20260926 21:26:32.790336 29829 tablet_bootstrap.cc:654] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:32.791404 29829 log.cc:824] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:32.792354 29830 tablet_bootstrap.cc:492] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Bootstrap starting.
I20260926 21:26:32.793211 29831 tablet_bootstrap.cc:492] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Bootstrap starting.
I20260926 21:26:32.796172 29829 tablet_bootstrap.cc:492] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: No bootstrap required, opened a new log
I20260926 21:26:32.796473 29829 ts_tablet_manager.cc:1397] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 21:26:32.798525 29831 tablet_bootstrap.cc:654] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:32.798540 29830 tablet_bootstrap.cc:654] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:32.799927 29830 log.cc:824] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:32.799927 29831 log.cc:824] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:32.803721 29830 tablet_bootstrap.cc:492] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: No bootstrap required, opened a new log
I20260926 21:26:32.803743 29831 tablet_bootstrap.cc:492] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: No bootstrap required, opened a new log
I20260926 21:26:32.804054 29830 ts_tablet_manager.cc:1397] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260926 21:26:32.804056 29831 ts_tablet_manager.cc:1397] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260926 21:26:32.804654 29829 raft_consensus.cc:348] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.805047 29829 raft_consensus.cc:374] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:32.805164 29829 raft_consensus.cc:729] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e86bff957846b280f82ad99567500e, State: Initialized, Role: FOLLOWER
I20260926 21:26:32.805783 29829 consensus_queue.cc:260] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.807617 29829 ts_tablet_manager.cc:1428] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260926 21:26:32.808352 29812 heartbeater.cc:503] Master 127.14.142.62:46307 was elected leader, sending a full tablet report...
I20260926 21:26:32.815160 29830 raft_consensus.cc:348] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.815677 29830 raft_consensus.cc:374] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:32.815778 29830 raft_consensus.cc:729] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f82abb48a9f4f0e8c8fb39aa2231aff, State: Initialized, Role: FOLLOWER
I20260926 21:26:32.816444 29830 consensus_queue.cc:260] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.817618 29831 raft_consensus.cc:348] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.818059 29831 raft_consensus.cc:374] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:32.818171 29831 raft_consensus.cc:729] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6ecda5b7d7a4b28922c981d065df503, State: Initialized, Role: FOLLOWER
I20260926 21:26:32.818619 29830 ts_tablet_manager.cc:1428] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 21:26:32.818811 29831 consensus_queue.cc:260] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.820640 29831 ts_tablet_manager.cc:1428] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260926 21:26:32.820717 29680 heartbeater.cc:503] Master 127.14.142.62:46307 was elected leader, sending a full tablet report...
W20260926 21:26:32.889147 29549 tablet.cc:2369] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:26:32.906391 29681 tablet.cc:2369] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:26:32.921021 29813 tablet.cc:2369] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:32.966665 29835 raft_consensus.cc:482] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:26:32.966939 29835 raft_consensus.cc:504] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.968767 29835 leader_election.cc:288] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919), b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2:44565)
I20260926 21:26:32.976408 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.969023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35919 (local address 127.14.142.3:39595)
0926 21:26:32.969228 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.969256 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.969417 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.970212 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.970220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.970268 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:32.970537 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.970554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.972019 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.972024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.973665 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.973678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.975536 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.975542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.975745 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.975786 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.975922 (+ 136us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.976106 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 21:26:32.976457 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.969374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:39595 (local address 127.14.142.1:35919)
0926 21:26:32.969700 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.969707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.969735 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:26:32.969846 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.969852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.969957 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.970103 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:32.970738 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.971828 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.973860 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.975362 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.975518 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.975589 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.976042 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.976225 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 21:26:32.977454 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.969764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:44565 (local address 127.14.142.3:59161)
0926 21:26:32.970731 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:32.970746 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:32.970877 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:32.971217 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:32.971222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:32.971244 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:32.971567 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.971575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.972956 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.972959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.974913 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:32.974940 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.976634 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.976640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:32.976883 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.976929 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 21:26:32.977057 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 21:26:32.977197 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":579,"threads_started":1}
I20260926 21:26:32.977517 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:32.969574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:59161 (local address 127.14.142.2:44565)
0926 21:26:32.969779 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:32.969783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:32.970829 (+ 1046us) server_negotiation.cc:407] Connection header received
0926 21:26:32.971038 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:32.971043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:32.971103 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:32.971207 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:32.971718 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.972791 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.975127 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:32.976352 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:32.976642 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:32.976725 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:32.977125 (+ 400us) server_negotiation.cc:299] Negotiation successful
0926 21:26:32.977257 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 21:26:32.977806 29502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8932db7da88f45f381fccd4bca66bca6" candidate_uuid: "37e86bff957846b280f82ad99567500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" is_pre_election: true
I20260926 21:26:32.978408 29502 raft_consensus.cc:2454] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e86bff957846b280f82ad99567500e in term 0.
I20260926 21:26:32.978523 29635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8932db7da88f45f381fccd4bca66bca6" candidate_uuid: "37e86bff957846b280f82ad99567500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ecda5b7d7a4b28922c981d065df503" is_pre_election: true
I20260926 21:26:32.978986 29635 raft_consensus.cc:2454] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e86bff957846b280f82ad99567500e in term 0.
I20260926 21:26:32.979089 29701 leader_election.cc:302] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e86bff957846b280f82ad99567500e, 6f82abb48a9f4f0e8c8fb39aa2231aff; no voters:
I20260926 21:26:32.979545 29835 raft_consensus.cc:2781] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:26:32.979737 29835 raft_consensus.cc:482] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:26:32.979893 29835 raft_consensus.cc:3037] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:32.982169 29835 raft_consensus.cc:504] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.982805 29835 leader_election.cc:288] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [CANDIDATE]: Term 1 election: Requested vote from peers 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919), b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2:44565)
I20260926 21:26:32.983819 29635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8932db7da88f45f381fccd4bca66bca6" candidate_uuid: "37e86bff957846b280f82ad99567500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6ecda5b7d7a4b28922c981d065df503"
I20260926 21:26:32.984040 29635 raft_consensus.cc:3037] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:32.984136 29502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8932db7da88f45f381fccd4bca66bca6" candidate_uuid: "37e86bff957846b280f82ad99567500e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
I20260926 21:26:32.984650 29502 raft_consensus.cc:3037] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:32.986532 29635 raft_consensus.cc:2454] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e86bff957846b280f82ad99567500e in term 1.
I20260926 21:26:32.987214 29703 leader_election.cc:302] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e86bff957846b280f82ad99567500e, b6ecda5b7d7a4b28922c981d065df503; no voters:
I20260926 21:26:32.987703 29502 raft_consensus.cc:2454] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e86bff957846b280f82ad99567500e in term 1.
I20260926 21:26:32.987735 29835 raft_consensus.cc:2781] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:32.988422 29835 raft_consensus.cc:686] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 1 LEADER]: Becoming Leader. State: Replica: 37e86bff957846b280f82ad99567500e, State: Running, Role: LEADER
I20260926 21:26:32.989076 29835 consensus_queue.cc:237] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } }
I20260926 21:26:32.995230 29358 catalog_manager.cc:5495] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e reported cstate change: term changed from 0 to 1, leader changed from <none> to 37e86bff957846b280f82ad99567500e (127.14.142.3). New cstate: current_term: 1 leader_uuid: "37e86bff957846b280f82ad99567500e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37e86bff957846b280f82ad99567500e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 44103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 } health_report { overall_health: UNKNOWN } } }
I20260926 21:26:33.061493 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:33.053196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.14.142.3:44103)
0926 21:26:33.053528 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:33.053539 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:33.053572 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:26:33.053939 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:33.053948 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:33.054117 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:33.054380 (+ 263us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:26:33.054943 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.056143 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.058304 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.059007 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.059160 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:33.059963 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:26:33.060771 (+ 808us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:26:33.060945 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:33.061027 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 21:26:33.061133 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 21:26:33.070350 29635 raft_consensus.cc:1261] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Refusing update from remote peer 37e86bff957846b280f82ad99567500e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:33.070358 29502 raft_consensus.cc:1261] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Refusing update from remote peer 37e86bff957846b280f82ad99567500e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:33.071381 29839 consensus_queue.cc:1035] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:33.071762 29835 consensus_queue.cc:1035] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6ecda5b7d7a4b28922c981d065df503" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 44565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:33.095428 29843 mvcc.cc:205] Tried to move back new op lower bound from 7333715939605716992 to 7333715939288117248. Current Snapshot: MvccSnapshot[applied={T|T < 7333715939605716992}]
I20260926 21:26:33.109814 29358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:33.110291 29358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:33.114290 29358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Sending DeleteTablet for 3 replicas of tablet 8932db7da88f45f381fccd4bca66bca6
I20260926 21:26:33.116113 29747 tablet_service.cc:1514] Processing DeleteTablet for tablet 8932db7da88f45f381fccd4bca66bca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:33 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:48372
I20260926 21:26:33.116165 29482 tablet_service.cc:1514] Processing DeleteTablet for tablet 8932db7da88f45f381fccd4bca66bca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:33 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:49782
I20260926 21:26:33.117610 29852 tablet_replica.cc:330] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: stopping tablet replica
I20260926 21:26:33.118183 29615 tablet_service.cc:1514] Processing DeleteTablet for tablet 8932db7da88f45f381fccd4bca66bca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:33 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:47222
I20260926 21:26:33.118322 29852 raft_consensus.cc:2229] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:33.119000 29852 raft_consensus.cc:2258] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:33.120985 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29398
I20260926 21:26:33.121726 29852 ts_tablet_manager.cc:1905] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:33.132579 29853 tablet_replica.cc:330] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: stopping tablet replica
I20260926 21:26:33.133812 29853 raft_consensus.cc:2229] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:26:33.134765 29853 raft_consensus.cc:2258] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:33.137591 29853 ts_tablet_manager.cc:1905] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:33.143450 29854 tablet_replica.cc:330] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: stopping tablet replica
I20260926 21:26:33.143983 29854 raft_consensus.cc:2229] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:33.144631 29854 raft_consensus.cc:2258] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:33.146178 29852 ts_tablet_manager.cc:1918] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:33.146461 29852 log.cc:1196] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/8932db7da88f45f381fccd4bca66bca6
I20260926 21:26:33.146692 29853 ts_tablet_manager.cc:1918] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:33.146898 29853 log.cc:1196] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/8932db7da88f45f381fccd4bca66bca6
I20260926 21:26:33.147476 29852 ts_tablet_manager.cc:1939] T 8932db7da88f45f381fccd4bca66bca6 P 6f82abb48a9f4f0e8c8fb39aa2231aff: Deleting consensus metadata
I20260926 21:26:33.147591 29853 ts_tablet_manager.cc:1939] T 8932db7da88f45f381fccd4bca66bca6 P 37e86bff957846b280f82ad99567500e: Deleting consensus metadata
I20260926 21:26:33.149816 29347 catalog_manager.cc:4841] TS 37e86bff957846b280f82ad99567500e (127.14.142.3:44103): tablet 8932db7da88f45f381fccd4bca66bca6 (table test-table [id=a5ce987524fb4539a962c18eff1625f4]) successfully deleted
I20260926 21:26:33.149887 29345 catalog_manager.cc:4841] TS 6f82abb48a9f4f0e8c8fb39aa2231aff (127.14.142.1:35919): tablet 8932db7da88f45f381fccd4bca66bca6 (table test-table [id=a5ce987524fb4539a962c18eff1625f4]) successfully deleted
I20260926 21:26:33.153221 29854 ts_tablet_manager.cc:1905] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:33.155561 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:33.121942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.14.142.1:35919)
0926 21:26:33.122119 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:33.122126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:33.122502 (+ 376us) server_negotiation.cc:407] Connection header received
0926 21:26:33.122587 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:33.122596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:33.122704 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:33.122839 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:33.125482 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.126940 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.129627 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.130526 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.130675 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:33.144852 (+ 14177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:33.144874 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:33.144893 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:33.144944 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:33.150354 (+ 5410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:33.151492 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:33.151501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:33.151504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:33.151681 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:33.152213 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:33.152220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:33.152223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:33.152755 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:33.154772 (+ 2017us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:33.154937 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 21:26:33.155180 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 21:26:33.156399 29542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:33.174942 29854 ts_tablet_manager.cc:1918] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:33.175212 29854 log.cc:1196] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/8932db7da88f45f381fccd4bca66bca6
I20260926 21:26:33.176005 29854 ts_tablet_manager.cc:1939] T 8932db7da88f45f381fccd4bca66bca6 P b6ecda5b7d7a4b28922c981d065df503: Deleting consensus metadata
I20260926 21:26:33.178505 29347 catalog_manager.cc:4841] TS b6ecda5b7d7a4b28922c981d065df503 (127.14.142.2:44565): tablet 8932db7da88f45f381fccd4bca66bca6 (table test-table [id=a5ce987524fb4539a962c18eff1625f4]) successfully deleted
I20260926 21:26:33.257053 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29398
I20260926 21:26:33.278229 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29552
I20260926 21:26:33.295214 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:33.279166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.14.142.2:44565)
0926 21:26:33.279366 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:33.279372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:33.279396 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:26:33.279501 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:33.279508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:33.279615 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:33.279736 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:33.281156 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.282476 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.283917 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.284828 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.284980 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:33.287776 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:33.287798 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:33.287819 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:33.287871 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:33.291780 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:33.292808 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:33.292819 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:33.292823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:33.292994 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:33.293414 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:33.293419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:33.293423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:33.293951 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:33.294239 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:33.294592 (+ 353us) server_negotiation.cc:299] Negotiation successful
0926 21:26:33.294828 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 21:26:33.296005 29675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:33.387796 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29552
I20260926 21:26:33.408862 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29684
I20260926 21:26:33.427285 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:33.409675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.14.142.3:44103)
0926 21:26:33.409902 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:33.409909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:33.409930 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:26:33.410068 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:33.410074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:33.410173 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:33.410286 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:33.411569 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.412846 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.414195 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:33.414857 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:33.415008 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:33.418454 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:33.418472 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:33.418496 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:33.418559 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:33.423149 (+ 4590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:33.424070 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:33.424085 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:33.424089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:33.424321 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:33.425224 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:33.425230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:33.425233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:33.425897 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:33.426208 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:33.426536 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 21:26:33.426807 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 21:26:33.428499 29807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:33.559823 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29684
I20260926 21:26:33.585729 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29327
I20260926 21:26:33.587203 29390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:33.703670 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29327
I20260926 21:26:33.727914 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46307
--webserver_interface=127.14.142.62
--webserver_port=46245
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:34.097894 29859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:34.098213 29859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:34.098305 29859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:34.105995 29859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:34.106117 29859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:34.106163 29859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:34.106215 29859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:34.106253 29859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:34.116482 29859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=46245
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:34.117846 29859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:34.119131 29859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:34.131989 29864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:34.131911 29865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:34.131836 29867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:34.133785 29859 server_base.cc:1034] running on GCE node
I20260926 21:26:34.135249 29859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:34.137647 29859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:34.138998 29859 hybrid_clock.cc:648] HybridClock initialized: now 1790457994138914 us; error 73 us; skew 500 ppm
I20260926 21:26:34.139467 29859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:34 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457994, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:34.143213 29859 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:34.145252 29859 webserver.cc:460] Webserver started at http://127.14.142.62:46245/ using document root <none> and password file <none>
I20260926 21:26:34.146075 29859 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:34.146248 29859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:34.152714 29859 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260926 21:26:34.155958 29874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:34.157500 29859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:26:34.157712 29859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "afb6f33edcaf4c3b9266dd6eb3145547"
format_stamp: "Formatted at 2026-09-26 21:26:27 on dist-test-slave-4d5t"
server_key: "e60b1f5cb0ab98d2418ff0bef754c9d8"
server_key_iv: "656e346a21870e3c977abaff0c852dd0"
server_key_version: "encryptionkey@0"
I20260926 21:26:34.158774 29859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:34.178422 29859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:34.185251 29859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:34.185618 29859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:34.208395 29859 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:46307
I20260926 21:26:34.208415 29925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:46307 every 8 connection(s)
I20260926 21:26:34.210115 29859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:34.217535 29926 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:26:34.217839 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29859
I20260926 21:26:34.218881 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:35919
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:46307
--webserver_port=42483
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:34.222918 29926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap starting.
I20260926 21:26:34.247999 29926 log.cc:824] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:34.265923 29926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:26:34.266678 29926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547: Bootstrap complete.
I20260926 21:26:34.278118 29926 raft_consensus.cc:348] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:34.279151 29926 raft_consensus.cc:729] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb6f33edcaf4c3b9266dd6eb3145547, State: Initialized, Role: FOLLOWER
I20260926 21:26:34.279860 29926 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:34.280143 29926 raft_consensus.cc:388] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:34.280310 29926 raft_consensus.cc:482] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:34.280534 29926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 2 FOLLOWER]: Advancing to term 3
I20260926 21:26:34.285907 29926 raft_consensus.cc:504] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:34.286512 29926 leader_election.cc:302] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afb6f33edcaf4c3b9266dd6eb3145547; no voters:
I20260926 21:26:34.287273 29926 leader_election.cc:288] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 21:26:34.287415 29930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 21:26:34.288534 29930 raft_consensus.cc:686] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [term 3 LEADER]: Becoming Leader. State: Replica: afb6f33edcaf4c3b9266dd6eb3145547, State: Running, Role: LEADER
I20260926 21:26:34.289247 29930 consensus_queue.cc:237] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } }
I20260926 21:26:34.289656 29926 sys_catalog.cc:564] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:34.292023 29932 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afb6f33edcaf4c3b9266dd6eb3145547. Latest consensus state: current_term: 3 leader_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } } }
I20260926 21:26:34.292380 29932 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:34.293035 29931 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f33edcaf4c3b9266dd6eb3145547" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 46307 } } }
I20260926 21:26:34.293332 29931 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f33edcaf4c3b9266dd6eb3145547 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:34.302538 29935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:34.319156 29935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:34.321730 29935 catalog_manager.cc:1259] Loaded cluster ID: f776696d4cc04675810ee4479f4a1a2d
I20260926 21:26:34.321947 29935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 21:26:34.324302 29935 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f22a16719ad google::LogMessage::Fail() at ??:0
I20260926 21:26:34.329025 29947 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f22a167488a google::LogMessage::SendToLog() at ??:0
@ 0x7f22a16714de google::LogMessage::Flush() at ??:0
@ 0x7f22a1672a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f22b70a4512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f22b70a1976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f22b7129610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f22b7129439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f22a42bd345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f22a42c091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f22a42c0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f22a429beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f22a75756db start_thread at ??:0
@ 0x7f22a050b61f clone at ??:0
W20260926 21:26:34.618824 29928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:34.619189 29928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:34.619362 29928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:34.627190 29928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:34.627362 29928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:34.627547 29928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:26:34.638187 29928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:35919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=42483
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:34.639591 29928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:34.640964 29928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:34.654351 29968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:34.654394 29971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:34.654438 29969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:34.656080 29928 server_base.cc:1034] running on GCE node
I20260926 21:26:34.656816 29928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:34.657776 29928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:34.659020 29928 hybrid_clock.cc:648] HybridClock initialized: now 1790457994658960 us; error 68 us; skew 500 ppm
I20260926 21:26:34.659396 29928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:34 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457994, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:34.662843 29928 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:26:34.664778 29928 webserver.cc:460] Webserver started at http://127.14.142.1:42483/ using document root <none> and password file <none>
I20260926 21:26:34.665562 29928 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:34.665726 29928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:34.671955 29928 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 21:26:34.675554 29978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:34.677209 29928 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 21:26:34.677417 29928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "6f82abb48a9f4f0e8c8fb39aa2231aff"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "dd105549cbc8c6a743a54315c8971d2e"
server_key_iv: "1c014ecbf6ab729de067f1d9c9e38a25"
server_key_version: "encryptionkey@0"
I20260926 21:26:34.678483 29928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:34.703055 29928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:34.708626 29928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:34.708982 29928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:34.710223 29928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:34.711799 29928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:34.711951 29928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:34.712105 29928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:34.712205 29928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:34.752039 29928 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:35919
I20260926 21:26:34.752134 30090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:35919 every 8 connection(s)
I20260926 21:26:34.753686 29928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:26:34.758497 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:34.756669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:33641)
0926 21:26:34.757426 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:34.757664 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":319,"threads_started":1}
W20260926 21:26:34.759410 30091 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:46307 (0 consecutive failures): Network error: Failed to ping master at 127.14.142.62:46307: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
I20260926 21:26:34.759454 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 29928
I20260926 21:26:34.760243 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:44565
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:46307
--webserver_port=37645
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:26:34.760659 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:34.760181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:58691)
0926 21:26:34.760370 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:34.760470 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 21:26:34.761648 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:34.761277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:34295)
0926 21:26:34.761435 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:34.761528 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 21:26:34.761933 30091 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 21:26:35.112293 30094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:35.112694 30094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:35.112802 30094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:35.120460 30094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:35.120685 30094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:35.120824 30094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:26:35.131057 30094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:44565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=37645
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:35.132467 30094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:35.133919 30094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:35.148105 30102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:35.148092 30100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:35.153106 30099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:35.153725 30094 server_base.cc:1034] running on GCE node
I20260926 21:26:35.154433 30094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:35.155577 30094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:35.156843 30094 hybrid_clock.cc:648] HybridClock initialized: now 1790457995156771 us; error 51 us; skew 500 ppm
I20260926 21:26:35.157228 30094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:35 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457995, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:35.160774 30094 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:26:35.162477 30094 webserver.cc:460] Webserver started at http://127.14.142.2:37645/ using document root <none> and password file <none>
I20260926 21:26:35.163260 30094 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:35.163429 30094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:35.170265 30094 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:26:35.173530 30109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:35.175130 30094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:26:35.175359 30094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "b6ecda5b7d7a4b28922c981d065df503"
format_stamp: "Formatted at 2026-09-26 21:26:28 on dist-test-slave-4d5t"
server_key: "8851d99afa9c0ac1670150ca43a041f7"
server_key_iv: "d5cb575e72b0480e377a4383c67c0d5d"
server_key_version: "encryptionkey@0"
I20260926 21:26:35.176888 30094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:35.203866 30094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:35.208580 30094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:35.208870 30094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:35.209952 30094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:35.211398 30094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:35.211524 30094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:35.211661 30094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:35.211758 30094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:35.247851 30094 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:44565
I20260926 21:26:35.247942 30222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:44565 every 8 connection(s)
I20260926 21:26:35.249382 30094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:26:35.254225 30225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.252304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:48559)
0926 21:26:35.253095 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.253338 (+ 243us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":346,"threads_started":1}
W20260926 21:26:35.254931 30223 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:46307 (0 consecutive failures): Network error: Failed to ping master at 127.14.142.62:46307: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
I20260926 21:26:35.255831 30225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.255474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:37955)
0926 21:26:35.255646 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.255717 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 21:26:35.256788 30225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.256385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:35407)
0926 21:26:35.256544 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.256673 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260926 21:26:35.257057 30223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 21:26:35.257725 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30094
I20260926 21:26:35.258335 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:44103
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:46307
--webserver_port=36919
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:35.608786 30226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:35.609087 30226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:35.609287 30226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:35.616767 30226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:35.616889 30226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:35.617063 30226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:26:35.627781 30226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:44103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=36919
--tserver_master_addrs=127.14.142.62:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:35.629277 30226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:35.630638 30226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:35.644147 30231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:35.644457 30234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:35.644496 30232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:35.645553 30226 server_base.cc:1034] running on GCE node
I20260926 21:26:35.646288 30226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:35.647363 30226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:35.648675 30226 hybrid_clock.cc:648] HybridClock initialized: now 1790457995648604 us; error 58 us; skew 500 ppm
I20260926 21:26:35.649027 30226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:35 dist-test-slave-4d5t krb5kdc[28774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790457995, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:35.652486 30226 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:26:35.654402 30226 webserver.cc:460] Webserver started at http://127.14.142.3:36919/ using document root <none> and password file <none>
I20260926 21:26:35.655157 30226 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:35.655316 30226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:35.661209 30226 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 21:26:35.664013 30241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:35.665861 30226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:26:35.666097 30226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "37e86bff957846b280f82ad99567500e"
format_stamp: "Formatted at 2026-09-26 21:26:29 on dist-test-slave-4d5t"
server_key: "892d53fc0ca821547b353d6a7464ffd9"
server_key_iv: "47b3b6650863f63e0bc812fbfa3494c9"
server_key_version: "encryptionkey@0"
I20260926 21:26:35.667125 30226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:35.691330 30226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:35.696280 30226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:35.696691 30226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:35.697846 30226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:35.699285 30226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:35.699422 30226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:35.699553 30226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:35.699644 30226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:35.737807 30226 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:44103
I20260926 21:26:35.737897 30353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:44103 every 8 connection(s)
I20260926 21:26:35.739254 30226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:26:35.744179 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.742274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:33945)
0926 21:26:35.743000 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.743225 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":294,"threads_started":1}
W20260926 21:26:35.745103 30354 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:46307 (0 consecutive failures): Network error: Failed to ping master at 127.14.142.62:46307: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
I20260926 21:26:35.745182 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30226
I20260926 21:26:35.746457 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.745930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:41821)
0926 21:26:35.746195 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.746301 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260926 21:26:35.747720 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.747251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:35817)
0926 21:26:35.747481 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.747593 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260926 21:26:35.747988 30354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 21:26:35.763445 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:35.762584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:50501)
0926 21:26:35.763085 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:35.763204 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":240,"threads_started":1}
I20260926 21:26:36.258471 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:36.257662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:54273)
0926 21:26:36.258140 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:36.258243 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":228,"threads_started":1}
I20260926 21:26:36.749893 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:36.748917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:45925)
0926 21:26:36.749520 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:36.749653 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":293,"threads_started":1}
I20260926 21:26:36.765216 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:36.764443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:52321)
0926 21:26:36.764956 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:36.765043 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":285,"threads_started":1}
I20260926 21:26:37.260344 30362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:37.259560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:36031)
0926 21:26:37.260080 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:37.260171 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":279,"threads_started":1}
I20260926 21:26:37.751974 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:37.751098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:46815)
0926 21:26:37.751624 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:37.751780 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":237,"threads_started":1}
I20260926 21:26:37.767105 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:37.766326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:52555)
0926 21:26:37.766799 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:37.766894 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":247,"threads_started":1}
I20260926 21:26:38.262661 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:38.261718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:33309)
0926 21:26:38.262352 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:38.262472 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":337,"threads_started":1}
I20260926 21:26:38.754057 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:38.753121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:50385)
0926 21:26:38.753766 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:38.753863 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":296,"threads_started":1}
I20260926 21:26:38.769013 30367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:38.768139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:54177)
0926 21:26:38.768738 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:38.768827 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":299,"threads_started":1}
I20260926 21:26:39.264635 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:39.263780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:56111)
0926 21:26:39.264289 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:39.264377 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":257,"threads_started":1}
I20260926 21:26:39.755807 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:39.755050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:50309)
0926 21:26:39.755546 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:39.755639 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":271,"threads_started":1}
I20260926 21:26:39.771114 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:39.770270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:45991)
0926 21:26:39.770831 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:39.770932 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":269,"threads_started":1}
I20260926 21:26:40.266850 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:40.265851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:49663)
0926 21:26:40.266517 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:40.266638 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":440,"threads_started":1}
I20260926 21:26:40.758320 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:40.757392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:36039)
0926 21:26:40.758022 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:40.758136 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":310,"threads_started":1}
I20260926 21:26:40.773263 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:40.772238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:40205)
0926 21:26:40.772919 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:40.773053 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":360,"threads_started":1}
I20260926 21:26:41.269474 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:41.268250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:35977)
0926 21:26:41.268939 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:41.269065 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":403,"threads_started":1}
I20260926 21:26:41.760545 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:41.759541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:48655)
0926 21:26:41.760229 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:41.760338 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":423,"threads_started":1}
I20260926 21:26:41.775748 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:41.774573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:43125)
0926 21:26:41.775311 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:41.775439 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":243,"threads_started":1}
I20260926 21:26:42.271870 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:42.270723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:47643)
0926 21:26:42.271494 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:42.271596 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":355,"threads_started":1}
I20260926 21:26:42.762522 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:42.761656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:37373)
0926 21:26:42.762235 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:42.762318 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":300,"threads_started":1}
I20260926 21:26:42.778142 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:42.777315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:43799)
0926 21:26:42.777868 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:42.777960 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
I20260926 21:26:43.274210 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:43.273152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:38653)
0926 21:26:43.273797 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:43.273936 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":286,"threads_started":1}
I20260926 21:26:43.764688 30381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:43.763635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:39019)
0926 21:26:43.764324 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:43.764439 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":466,"threads_started":1}
I20260926 21:26:43.779968 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:43.779123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:53375)
0926 21:26:43.779710 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:43.779791 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":347,"threads_started":1}
I20260926 21:26:44.276136 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:44.275245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:49139)
0926 21:26:44.275848 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:44.275942 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":250,"threads_started":1}
I20260926 21:26:44.767035 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:44.765883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:44133)
0926 21:26:44.766622 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:44.766734 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":365,"threads_started":1}
I20260926 21:26:44.781838 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:44.781009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:46605)
0926 21:26:44.781591 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:44.781676 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":346,"threads_started":1}
I20260926 21:26:45.278007 30386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:45.277278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:56443)
0926 21:26:45.277755 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:45.277842 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":277,"threads_started":1}
I20260926 21:26:45.769017 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:45.768214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:45705)
0926 21:26:45.768760 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:45.768840 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":264,"threads_started":1}
I20260926 21:26:45.783707 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:45.782837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:53247)
0926 21:26:45.783407 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:45.783512 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":310,"threads_started":1}
I20260926 21:26:46.279837 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:46.279012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:51559)
0926 21:26:46.279475 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:46.279591 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260926 21:26:46.771147 30390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:46.770093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:55247)
0926 21:26:46.770802 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:46.770921 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":434,"threads_started":1}
I20260926 21:26:46.785542 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:46.784715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:44329)
0926 21:26:46.785268 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:46.785361 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":318,"threads_started":1}
I20260926 21:26:47.282199 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:47.281188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:47049)
0926 21:26:47.281914 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:47.282010 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":359,"threads_started":1}
I20260926 21:26:47.773825 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:47.772541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:36897)
0926 21:26:47.773496 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:47.773613 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":606,"threads_started":1}
I20260926 21:26:47.787503 30394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:47.786638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:56959)
0926 21:26:47.787183 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:47.787291 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":293,"threads_started":1}
I20260926 21:26:48.284438 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:48.283425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:53111)
0926 21:26:48.284054 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:48.284205 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":286,"threads_started":1}
I20260926 21:26:48.776450 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:48.774990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:51321)
0926 21:26:48.775854 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:48.776070 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":295,"threads_started":1}
I20260926 21:26:48.789767 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:48.788677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:49589)
0926 21:26:48.789339 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:48.789486 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":316,"threads_started":1}
I20260926 21:26:49.286171 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:49.285345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:38365)
0926 21:26:49.285896 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:49.285983 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":335,"threads_started":1}
I20260926 21:26:49.778662 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:49.777665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:46969)
0926 21:26:49.778281 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:49.778412 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
I20260926 21:26:49.792176 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:49.791326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:41067)
0926 21:26:49.791822 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:49.791947 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
I20260926 21:26:50.288921 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:50.287693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.2:36445)
0926 21:26:50.288491 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:50.288712 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":422,"threads_started":1}
I20260926 21:26:50.759168 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 29928
I20260926 21:26:50.781993 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:50.780272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.3:59063)
0926 21:26:50.781609 (+ 1337us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:50.781802 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1205,"thread_start_us":957,"threads_started":1}
I20260926 21:26:50.794477 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:50.793503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:46307 (local address 127.14.142.1:34615)
0926 21:26:50.794189 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:50.794311 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":335,"threads_started":1}
I20260926 21:26:50.891433 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:50.760438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.14.142.1:35919)
0926 21:26:50.761076 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:50.761093 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:50.873447 (+112354us) server_negotiation.cc:407] Connection header received
0926 21:26:50.875332 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:50.875341 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:50.875786 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:50.876148 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:50.877781 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:50.879017 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:50.880358 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:50.881191 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:50.881396 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:50.884351 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:50.884360 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:50.884378 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:50.884418 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:50.887723 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:50.888602 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:50.888613 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:50.888625 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:50.888896 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:50.889321 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:50.889327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:50.889331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:50.889888 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:50.890204 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:50.890676 (+ 472us) server_negotiation.cc:299] Negotiation successful
0926 21:26:50.891126 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":302,"threads_started":1}
I20260926 21:26:50.892951 30086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:50.980520 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 29928
I20260926 21:26:50.999488 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30094
I20260926 21:26:51.020920 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:51.000504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.14.142.2:44565)
0926 21:26:51.001114 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:51.001127 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:51.001164 (+ 37us) server_negotiation.cc:407] Connection header received
0926 21:26:51.002775 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:51.002785 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:51.003222 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:51.003513 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:51.005486 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:51.006898 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:51.008254 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:51.009058 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:51.009199 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:51.012055 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:51.012067 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:51.012092 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:51.012164 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:51.015813 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:51.016977 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:51.016987 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:51.016997 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:51.017394 (+ 397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:51.018117 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:51.018121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:51.018124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:51.018902 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:51.019171 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:51.019713 (+ 542us) server_negotiation.cc:299] Negotiation successful
0926 21:26:51.020520 (+ 807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":278,"threads_started":1}
I20260926 21:26:51.022643 30218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:51.112871 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30094
I20260926 21:26:51.131546 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30226
I20260926 21:26:51.151350 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:51.132937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.14.142.3:44103)
0926 21:26:51.133494 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:51.133509 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:51.133550 (+ 41us) server_negotiation.cc:407] Connection header received
0926 21:26:51.135248 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:51.135258 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:51.135700 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:51.136076 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:51.137667 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:51.138887 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:51.140228 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:51.140964 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:51.141159 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:51.144190 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:51.144200 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:51.144219 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:51.144270 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:51.147612 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:51.148519 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:51.148526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:51.148537 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:51.148789 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:51.149194 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:51.149205 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:51.149207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:51.149789 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:51.150095 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:51.150546 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 21:26:51.151030 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":244,"threads_started":1}
I20260926 21:26:51.152930 30349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:51.238253 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30226
2026-09-26T21:26:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25340 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:26:51 dist-test-slave-4d5t krb5kdc[30412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:26:51 dist-test-slave-4d5t krb5kdc[30412](info): set up 2 sockets
Sep 26 21:26:51 dist-test-slave-4d5t krb5kdc[30412](info): commencing operation
krb5kdc: starting...
W20260926 21:26:52.401561 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:26:52 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:26:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:26:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:26:52.672469 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36153
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:53.027201 30425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:53.027576 30425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:53.027742 30425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:53.035833 30425 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 21:26:53.036000 30425 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 21:26:53.036101 30425 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 21:26:53.036186 30425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:26:53.036289 30425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:53.036410 30425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:53.036496 30425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:26:53.036600 30425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:26:53.047230 30425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:36153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:53.048789 30425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:53.050230 30425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:53.069676 30430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:53.069729 30431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:53.069706 30433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:53.070791 30425 server_base.cc:1034] running on GCE node
I20260926 21:26:53.071785 30425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:53.073529 30425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:53.074827 30425 hybrid_clock.cc:648] HybridClock initialized: now 1790458013074761 us; error 38 us; skew 500 ppm
I20260926 21:26:53.075246 30425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:53 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458013, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:53.079123 30425 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:26:53.080955 30425 webserver.cc:460] Webserver started at http://127.14.142.62:45111/ using document root <none> and password file <none>
I20260926 21:26:53.081717 30425 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:53.081853 30425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:53.082192 30425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:53.085533 30425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "9f708d0b859244e3b2c8b67004fe8fb2"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "f40046ef5f30ebf82d675d43417b6d2b"
server_key_iv: "618a45f0e8113446b54db1930548ed43"
server_key_version: "encryptionkey@0"
I20260926 21:26:53.086421 30425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "9f708d0b859244e3b2c8b67004fe8fb2"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "f40046ef5f30ebf82d675d43417b6d2b"
server_key_iv: "618a45f0e8113446b54db1930548ed43"
server_key_version: "encryptionkey@0"
I20260926 21:26:53.093196 30425 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:26:53.097131 30440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:53.098997 30425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:26:53.099326 30425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "9f708d0b859244e3b2c8b67004fe8fb2"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "f40046ef5f30ebf82d675d43417b6d2b"
server_key_iv: "618a45f0e8113446b54db1930548ed43"
server_key_version: "encryptionkey@0"
I20260926 21:26:53.099601 30425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:53.121296 30425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:53.125861 30425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:53.126156 30425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:53.149130 30425 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:36153
I20260926 21:26:53.149137 30491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:36153 every 8 connection(s)
I20260926 21:26:53.150570 30425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:26:53.153337 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30425
I20260926 21:26:53.153584 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:26:53.154408 14904 external_mini_cluster.cc:1428] Setting key de2a6cc5751ac1d2074d77696b514701
I20260926 21:26:53.155200 30492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:53.171255 30492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap starting.
Sep 26 21:26:53 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:53.175631 30492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:53.176918 30492 log.cc:824] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:53.179939 30492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: No bootstrap required, opened a new log
I20260926 21:26:53.182183 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:53.156657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.14.142.62:36153)
0926 21:26:53.157557 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:53.157570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:53.157611 (+ 41us) server_negotiation.cc:407] Connection header received
0926 21:26:53.159908 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:53.159932 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:53.160485 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:53.160902 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:53.163383 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:53.165244 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:53.167297 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:53.168355 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:53.168525 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:53.173734 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:53.173743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:53.173760 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:53.173804 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:53.177553 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:53.178464 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:53.178473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:53.178480 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:53.178670 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:53.179246 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:53.179258 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:53.179261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:53.179802 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:53.180136 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:53.180661 (+ 525us) server_negotiation.cc:299] Negotiation successful
0926 21:26:53.181117 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":426,"threads_started":1}
I20260926 21:26:53.189213 30492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:26:53.189669 30492 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:53.189785 30492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f708d0b859244e3b2c8b67004fe8fb2, State: Initialized, Role: FOLLOWER
I20260926 21:26:53.190363 30492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:26:53.190622 30492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:26:53.190778 30492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:26:53.190958 30492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:53.193553 30492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:26:53.194125 30492 leader_election.cc:302] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f708d0b859244e3b2c8b67004fe8fb2; no voters:
I20260926 21:26:53.194815 30492 leader_election.cc:288] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:26:53.195006 30497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:53.195798 30497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 9f708d0b859244e3b2c8b67004fe8fb2, State: Running, Role: LEADER
I20260926 21:26:53.196411 30497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:26:53.197214 30492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:26:53.199621 30498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } } }
I20260926 21:26:53.200143 30498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:53.200464 30499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f708d0b859244e3b2c8b67004fe8fb2. Latest consensus state: current_term: 1 leader_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } } }
I20260926 21:26:53.200824 30499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:26:53.204051 30506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:26:53.209511 30506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:26:53.219466 30506 catalog_manager.cc:1347] Generated new cluster ID: 78d59a37a4664c94a946612dbf5f81c9
I20260926 21:26:53.219728 30506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:26:53.244618 30506 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:26:53.246445 30506 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:26:53.254674 30506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Generated new TSK 0
I20260926 21:26:53.255863 30506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:26:53.325392 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36153
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:53.679916 30518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:53.680222 30518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:53.680384 30518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:53.688340 30518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:53.688472 30518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:53.688697 30518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:26:53.699123 30518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:53.700538 30518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:53.702090 30518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:53.716380 30523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:53.716934 30526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:53.716984 30524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:53.718127 30518 server_base.cc:1034] running on GCE node
I20260926 21:26:53.718762 30518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:53.719738 30518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:53.720989 30518 hybrid_clock.cc:648] HybridClock initialized: now 1790458013720937 us; error 44 us; skew 500 ppm
I20260926 21:26:53.721339 30518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:53 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458013, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:53.724545 30518 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:26:53.726176 30518 webserver.cc:460] Webserver started at http://127.14.142.1:35405/ using document root <none> and password file <none>
I20260926 21:26:53.726899 30518 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:53.727058 30518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:53.727375 30518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:53.730701 30518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "195ea3d896c04154a6a797e4ba5c4da7"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "cb78809b5ab0ac881ce8b191c0159590"
server_key_iv: "cb969eb0dfce71c62d354a81001f044d"
server_key_version: "encryptionkey@0"
I20260926 21:26:53.731582 30518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "195ea3d896c04154a6a797e4ba5c4da7"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "cb78809b5ab0ac881ce8b191c0159590"
server_key_iv: "cb969eb0dfce71c62d354a81001f044d"
server_key_version: "encryptionkey@0"
I20260926 21:26:53.737231 30518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 21:26:53.741130 30533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:53.742660 30518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 21:26:53.742874 30518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "195ea3d896c04154a6a797e4ba5c4da7"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "cb78809b5ab0ac881ce8b191c0159590"
server_key_iv: "cb969eb0dfce71c62d354a81001f044d"
server_key_version: "encryptionkey@0"
I20260926 21:26:53.743098 30518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:53.770480 30518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:53.775511 30518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:53.775878 30518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:53.777019 30518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:53.778502 30518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:53.778626 30518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:53.778788 30518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:53.778865 30518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:53.818657 30518 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46271
I20260926 21:26:53.818699 30645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46271 every 8 connection(s)
I20260926 21:26:53.820426 30518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:26:53.822634 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30518
I20260926 21:26:53.822881 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:26:53.823961 14904 external_mini_cluster.cc:1428] Setting key e152aab1709a86a236c29bbbea3fbfba
Sep 26 21:26:53 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458013, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:53.843477 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:53.823732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:42703 (local address 127.14.142.62:36153)
0926 21:26:53.824230 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:53.824235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:53.825661 (+ 1426us) server_negotiation.cc:407] Connection header received
0926 21:26:53.827682 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:53.827687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:53.827788 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:53.827908 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:53.829552 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:53.830297 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:53.831933 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:53.832345 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:53.832486 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:53.836892 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:53.836898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:53.836902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:53.836936 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:53.840113 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:53.841045 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:53.841053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:53.841062 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:53.841185 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:53.841641 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:53.841651 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:53.841653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:53.841936 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:53.842103 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:53.842849 (+ 746us) server_negotiation.cc:299] Negotiation successful
0926 21:26:53.843097 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":268,"threads_started":1}
I20260926 21:26:53.844533 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:53.824586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:42703)
0926 21:26:53.825468 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:53.825521 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:53.827403 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:53.828184 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:53.828193 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:53.828787 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:53.829376 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:53.829394 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:53.830463 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:53.830474 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:53.831774 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:53.831788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:53.832633 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:53.832641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:53.832876 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:53.833787 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:53.833824 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:53.836640 (+ 2816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:53.840304 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:53.840314 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:53.840334 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:53.840899 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:53.841306 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:53.841311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:53.841314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:53.841511 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:53.842086 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:53.842095 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:53.842454 (+ 359us) client_negotiation.cc:769] Sending connection context
0926 21:26:53.842736 (+ 282us) client_negotiation.cc:240] Negotiation successful
0926 21:26:53.843320 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":341,"threads_started":1}
I20260926 21:26:53.846400 30646 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36153
I20260926 21:26:53.846772 30646 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:53.847857 30646 heartbeater.cc:511] Master 127.14.142.62:36153 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
I20260926 21:26:53.850579 30457 ts_manager.cc:194] Registered new tserver with Master: 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
I20260926 21:26:53.852391 30457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:42703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:26:53.883816 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36153
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:54.239565 30652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:54.239856 30652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:54.240026 30652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:54.247411 30652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:54.247539 30652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:54.247727 30652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:26:54.258224 30652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:54.259586 30652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:54.261070 30652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:54.275596 30660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:54.275846 30657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:54.275550 30658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:54.277352 30652 server_base.cc:1034] running on GCE node
I20260926 21:26:54.278136 30652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:54.279193 30652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:54.280462 30652 hybrid_clock.cc:648] HybridClock initialized: now 1790458014280395 us; error 62 us; skew 500 ppm
I20260926 21:26:54.280970 30652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:54 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458014, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:54.284412 30652 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:26:54.286602 30652 webserver.cc:460] Webserver started at http://127.14.142.2:39213/ using document root <none> and password file <none>
I20260926 21:26:54.287649 30652 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:54.287791 30652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:54.288132 30652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:54.292075 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "202c9c8dd816ef4a137be667f187c309"
server_key_iv: "917a0f54a8a5a0df5779ec783aae4a05"
server_key_version: "encryptionkey@0"
I20260926 21:26:54.293102 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "202c9c8dd816ef4a137be667f187c309"
server_key_iv: "917a0f54a8a5a0df5779ec783aae4a05"
server_key_version: "encryptionkey@0"
I20260926 21:26:54.299285 30652 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:26:54.303730 30667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:54.305400 30652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:26:54.305554 30652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "202c9c8dd816ef4a137be667f187c309"
server_key_iv: "917a0f54a8a5a0df5779ec783aae4a05"
server_key_version: "encryptionkey@0"
I20260926 21:26:54.305738 30652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:54.318177 30652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:54.322717 30652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:54.322975 30652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:54.323922 30652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:54.325326 30652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:54.325402 30652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:54.325579 30652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:54.325673 30652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:54.362154 30652 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33011
I20260926 21:26:54.362200 30779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33011 every 8 connection(s)
I20260926 21:26:54.363518 30652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:26:54.372606 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30652
I20260926 21:26:54.372807 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:26:54.373324 14904 external_mini_cluster.cc:1428] Setting key 0a06b6a7f23cc5603951cc4ddbade923
Sep 26 21:26:54 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458014, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:54.387187 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:54.366219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:52179 (local address 127.14.142.62:36153)
0926 21:26:54.366684 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:54.366689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:54.367677 (+ 988us) server_negotiation.cc:407] Connection header received
0926 21:26:54.369766 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:54.369772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:54.370091 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:54.370209 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:54.371806 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.373036 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.375125 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.375797 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.376072 (+ 275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:54.379768 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:54.379778 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:54.379785 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:54.379838 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:54.383700 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:54.384740 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:54.384747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:54.384752 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:54.384950 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:54.385419 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:54.385424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:54.385428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:54.385708 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:54.385864 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:54.386577 (+ 713us) server_negotiation.cc:299] Negotiation successful
0926 21:26:54.386802 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260926 21:26:54.388206 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:54.366780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:52179)
0926 21:26:54.367503 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:54.367545 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:54.369343 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:54.370436 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:54.370444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:54.370998 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:54.371565 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:54.371582 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.373199 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.373202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:54.374916 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:54.374938 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.376042 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.376047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:54.376252 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:54.377234 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:54.377261 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:54.379481 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:54.383881 (+ 4400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:54.383893 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:54.383925 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:54.384601 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:54.385077 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:54.385083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:54.385087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:54.385289 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:54.385841 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:54.385851 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:54.386198 (+ 347us) client_negotiation.cc:769] Sending connection context
0926 21:26:54.386482 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 21:26:54.387043 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":285,"threads_started":1}
I20260926 21:26:54.389855 30780 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36153
I20260926 21:26:54.390565 30780 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:54.391901 30780 heartbeater.cc:511] Master 127.14.142.62:36153 requested a full tablet report, sending...
I20260926 21:26:54.393822 30457 ts_manager.cc:194] Registered new tserver with Master: 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011)
I20260926 21:26:54.394704 30457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:52179
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:26:54.436908 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36153
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:26:54.784451 30786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:26:54.784883 30786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:26:54.785049 30786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:26:54.792728 30786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:26:54.792858 30786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:26:54.793038 30786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:26:54.804535 30786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:26:54.806085 30786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:26:54.807528 30786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:26:54.822600 30792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:54.822850 30794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:26:54.822634 30791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:26:54.823690 30786 server_base.cc:1034] running on GCE node
I20260926 21:26:54.824441 30786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:26:54.825536 30786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:26:54.826776 30786 hybrid_clock.cc:648] HybridClock initialized: now 1790458014826720 us; error 38 us; skew 500 ppm
I20260926 21:26:54.827164 30786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:26:54 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458014, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:26:54.834574 30786 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:26:54.836462 30786 webserver.cc:460] Webserver started at http://127.14.142.3:32951/ using document root <none> and password file <none>
I20260926 21:26:54.837320 30786 fs_manager.cc:359] Metadata directory not provided
I20260926 21:26:54.837458 30786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:26:54.837821 30786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:26:54.841415 30786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "0604685c49f84fbca980887373eafd3e"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "b5147f55b79929b5f1f892a11220d920"
server_key_iv: "12ce50e119244b9724c35ca202b9f8ce"
server_key_version: "encryptionkey@0"
I20260926 21:26:54.842374 30786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "0604685c49f84fbca980887373eafd3e"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "b5147f55b79929b5f1f892a11220d920"
server_key_iv: "12ce50e119244b9724c35ca202b9f8ce"
server_key_version: "encryptionkey@0"
I20260926 21:26:54.849192 30786 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 21:26:54.853193 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:54.854796 30786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:26:54.855017 30786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "0604685c49f84fbca980887373eafd3e"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "b5147f55b79929b5f1f892a11220d920"
server_key_iv: "12ce50e119244b9724c35ca202b9f8ce"
server_key_version: "encryptionkey@0"
I20260926 21:26:54.855275 30786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:26:54.856148 30646 heartbeater.cc:503] Master 127.14.142.62:36153 was elected leader, sending a full tablet report...
I20260926 21:26:54.878010 30786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:26:54.883286 30786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:26:54.883569 30786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:26:54.884703 30786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:26:54.886096 30786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:26:54.886178 30786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:54.886263 30786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:26:54.886312 30786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:26:54.923243 30786 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:34999
I20260926 21:26:54.923287 30913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:34999 every 8 connection(s)
I20260926 21:26:54.924610 30786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:26:54.934077 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30786
I20260926 21:26:54.934324 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:26:54.934830 14904 external_mini_cluster.cc:1428] Setting key 9f3e557f9db3039fdbd2b88b380af30a
Sep 26 21:26:54 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458014, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:26:54.946085 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:54.927233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48091 (local address 127.14.142.62:36153)
0926 21:26:54.927769 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:54.927775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:54.928834 (+ 1059us) server_negotiation.cc:407] Connection header received
0926 21:26:54.930703 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:54.930708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:54.930841 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:54.930975 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:54.932454 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.933299 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.935109 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.935748 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.935905 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:54.939863 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:54.939868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:54.939871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:54.939906 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:54.942561 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:54.943356 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:54.943361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:54.943363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:54.943500 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:54.944046 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:54.944052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:54.944055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:54.944538 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:54.944880 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:54.945584 (+ 704us) server_negotiation.cc:299] Negotiation successful
0926 21:26:54.945799 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":225,"threads_started":1}
I20260926 21:26:54.947100 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:54.927733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:48091)
0926 21:26:54.928669 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:54.928718 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:54.930390 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:54.931178 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:54.931186 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:54.931707 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:26:54.932286 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:54.932304 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.933485 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.933490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:54.934904 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:54.934917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.936016 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.936025 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:54.936335 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:54.937106 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:26:54.937136 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:26:54.939628 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:26:54.942732 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:54.942740 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:54.942757 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:54.943218 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:54.943606 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:26:54.943609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:26:54.943612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:26:54.943827 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:26:54.944844 (+ 1017us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:26:54.944850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:26:54.945213 (+ 363us) client_negotiation.cc:769] Sending connection context
0926 21:26:54.945459 (+ 246us) client_negotiation.cc:240] Negotiation successful
0926 21:26:54.945895 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":286,"threads_started":1}
I20260926 21:26:54.949240 30914 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36153
I20260926 21:26:54.949754 30914 heartbeater.cc:464] Registering TS with master...
I20260926 21:26:54.950552 30914 heartbeater.cc:511] Master 127.14.142.62:36153 requested a full tablet report, sending...
I20260926 21:26:54.952309 30457 ts_manager.cc:194] Registered new tserver with Master: 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
I20260926 21:26:54.953235 30457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:48091
I20260926 21:26:54.963619 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:26:54.983529 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:54.968994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.14.142.62:36153)
0926 21:26:54.969191 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:54.969196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:54.969818 (+ 622us) server_negotiation.cc:407] Connection header received
0926 21:26:54.969995 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:54.970000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:54.970075 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:54.970206 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:54.971765 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.972656 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.974119 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:54.974536 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:54.974652 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:54.977294 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:54.977300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:54.977303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:54.977344 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:54.980181 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:54.981508 (+ 1327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:54.981513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:54.981515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:54.981672 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:54.982178 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:54.982182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:54.982186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:54.982559 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:54.982783 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:54.983080 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 21:26:54.983294 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:26:54.988066 30457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:26:54.996635 30457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:26:55.031275 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.016782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33011 (local address 127.0.0.1:37994)
0926 21:26:55.017331 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:55.017349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:55.017462 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:55.022405 (+ 4943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:55.022416 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:55.022463 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:55.022922 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.022939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.025036 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.025042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.027653 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.027674 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.029961 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.029967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.030235 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.030309 (+ 74us) client_negotiation.cc:769] Sending connection context
0926 21:26:55.030448 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 21:26:55.030701 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":1,"thread_start_us":753,"threads_started":1}
I20260926 21:26:55.031672 30928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.012174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:46706)
0926 21:26:55.013770 (+ 1596us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:55.013785 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:55.013864 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:55.025185 (+ 11321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:55.025190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:55.025222 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:55.025677 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.025689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.027367 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.027372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.028984 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.028997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.030653 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.030661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.030839 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.030873 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:26:55.030939 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 21:26:55.031072 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1496,"mutex_wait_us":46,"thread_start_us":1350,"threads_started":1}
I20260926 21:26:55.032702 30927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.011956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34999 (local address 127.0.0.1:56728)
0926 21:26:55.013196 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:55.013233 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:55.013410 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:55.023873 (+ 10463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:55.023881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:55.023920 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:55.024362 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.024375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.026431 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.026438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.029025 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.029043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.031627 (+ 2584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.031635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.031980 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.032015 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 21:26:55.032136 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 21:26:55.032284 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":1,"thread_start_us":828,"threads_started":1}
I20260926 21:26:55.032764 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.012469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.14.142.3:34999)
0926 21:26:55.023210 (+ 10741us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.023221 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.023258 (+ 37us) server_negotiation.cc:407] Connection header received
0926 21:26:55.023361 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.023371 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.023627 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.023844 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:55.024634 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.026266 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.029257 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.030880 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.031016 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.031315 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.032191 (+ 876us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.032369 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10525,"thread_start_us":10372,"threads_started":1}
I20260926 21:26:55.032757 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.012759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.14.142.1:46271)
0926 21:26:55.024335 (+ 11576us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.024345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.024373 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:26:55.024469 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.024479 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.024747 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.025866 (+ 1119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:55.026113 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.027208 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.029151 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.030406 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.031399 (+ 993us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.032049 (+ 650us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.032158 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.032369 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11353,"thread_start_us":12124,"threads_started":1}
I20260926 21:26:55.032941 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.017374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.14.142.2:33011)
0926 21:26:55.021740 (+ 4366us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.021758 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.021788 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:26:55.021888 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.021894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.022139 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.022387 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:55.023148 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.024869 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.027920 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.029709 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.032005 (+ 2296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.032226 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.032300 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.032501 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4187,"thread_start_us":4041,"threads_started":1}
I20260926 21:26:55.036518 30581 tablet_service.cc:1467] Processing CreateTablet for tablet ad8a5c18fe2343299822e8797f999b63 (DEFAULT_TABLE table=test-table [id=0ed7f5d74d5848dc890768f977c5dbd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:55.036520 30849 tablet_service.cc:1467] Processing CreateTablet for tablet ad8a5c18fe2343299822e8797f999b63 (DEFAULT_TABLE table=test-table [id=0ed7f5d74d5848dc890768f977c5dbd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:55.036437 30715 tablet_service.cc:1467] Processing CreateTablet for tablet ad8a5c18fe2343299822e8797f999b63 (DEFAULT_TABLE table=test-table [id=0ed7f5d74d5848dc890768f977c5dbd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:26:55.038069 30581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8a5c18fe2343299822e8797f999b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:55.038106 30715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8a5c18fe2343299822e8797f999b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:55.038368 30849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8a5c18fe2343299822e8797f999b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:26:55.050737 30933 tablet_bootstrap.cc:492] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Bootstrap starting.
I20260926 21:26:55.053730 30934 tablet_bootstrap.cc:492] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Bootstrap starting.
I20260926 21:26:55.057245 30934 tablet_bootstrap.cc:654] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:55.057243 30933 tablet_bootstrap.cc:654] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:55.058667 30934 log.cc:824] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:55.058667 30933 log.cc:824] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:55.059756 30935 tablet_bootstrap.cc:492] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Bootstrap starting.
I20260926 21:26:55.063446 30934 tablet_bootstrap.cc:492] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: No bootstrap required, opened a new log
I20260926 21:26:55.063560 30933 tablet_bootstrap.cc:492] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: No bootstrap required, opened a new log
I20260926 21:26:55.063804 30934 ts_tablet_manager.cc:1397] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 21:26:55.063848 30933 ts_tablet_manager.cc:1397] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260926 21:26:55.064038 30935 tablet_bootstrap.cc:654] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Neither blocks nor log segments found. Creating new log.
I20260926 21:26:55.065627 30935 log.cc:824] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Log is configured to *not* fsync() on all Append() calls
I20260926 21:26:55.069207 30935 tablet_bootstrap.cc:492] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: No bootstrap required, opened a new log
I20260926 21:26:55.069507 30935 ts_tablet_manager.cc:1397] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260926 21:26:55.073226 30934 raft_consensus.cc:348] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.073668 30934 raft_consensus.cc:374] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:55.073767 30934 raft_consensus.cc:729] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0604685c49f84fbca980887373eafd3e, State: Initialized, Role: FOLLOWER
I20260926 21:26:55.074424 30934 consensus_queue.cc:260] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.075639 30914 heartbeater.cc:503] Master 127.14.142.62:36153 was elected leader, sending a full tablet report...
I20260926 21:26:55.075919 30933 raft_consensus.cc:348] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.076542 30934 ts_tablet_manager.cc:1428] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 21:26:55.076606 30933 raft_consensus.cc:374] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:55.076720 30933 raft_consensus.cc:729] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a97f6079b3e4d3a869a78e4d857d6cb, State: Initialized, Role: FOLLOWER
I20260926 21:26:55.077615 30933 consensus_queue.cc:260] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.079928 30780 heartbeater.cc:503] Master 127.14.142.62:36153 was elected leader, sending a full tablet report...
I20260926 21:26:55.081110 30933 ts_tablet_manager.cc:1428] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260926 21:26:55.082970 30935 raft_consensus.cc:348] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.083698 30935 raft_consensus.cc:374] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:26:55.083848 30935 raft_consensus.cc:729] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195ea3d896c04154a6a797e4ba5c4da7, State: Initialized, Role: FOLLOWER
I20260926 21:26:55.084817 30935 consensus_queue.cc:260] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.087028 30935 ts_tablet_manager.cc:1428] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Time spent starting tablet: real 0.017s user 0.011s sys 0.006s
W20260926 21:26:55.115746 30781 tablet.cc:2369] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:26:55.176286 30915 tablet.cc:2369] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:26:55.219101 30939 raft_consensus.cc:482] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:26:55.219452 30939 raft_consensus.cc:504] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.221431 30939 leader_election.cc:288] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271), 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011)
I20260926 21:26:55.227857 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.221913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33011 (local address 127.14.142.3:58181)
0926 21:26:55.222380 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:55.222403 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:55.222603 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:55.222963 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:55.222969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:55.222999 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:55.223219 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.223227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.224278 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.224287 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.225769 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.225784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.226994 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.227000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.227212 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.227239 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 21:26:55.227360 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 21:26:55.227521 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":1,"spinlock_wait_cycles":22912}
I20260926 21:26:55.227839 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.222155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:58181 (local address 127.14.142.2:33011)
0926 21:26:55.222378 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.222384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.222506 (+ 122us) server_negotiation.cc:407] Connection header received
0926 21:26:55.222758 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.222767 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.222830 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.222944 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:55.223369 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.224107 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.225957 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.226815 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.226999 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.227079 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.227389 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.227530 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 21:26:55.228381 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.221888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:42597 (local address 127.14.142.1:46271)
0926 21:26:55.222187 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.222196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.223004 (+ 808us) server_negotiation.cc:407] Connection header received
0926 21:26:55.223220 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.223230 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.223300 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.223409 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:26:55.223975 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.224755 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.226529 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.227379 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.227592 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.227677 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.227966 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.228141 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260926 21:26:55.228826 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.221966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.14.142.3:42597)
0926 21:26:55.222865 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0926 21:26:55.222880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 21:26:55.223052 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:26:55.223423 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:26:55.223428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:26:55.223450 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:26:55.223793 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.223802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.224926 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.224930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.226352 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:26:55.226364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.227613 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.227619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:26:55.227797 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.227821 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 21:26:55.228305 (+ 484us) client_negotiation.cc:240] Negotiation successful
0926 21:26:55.228468 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":499,"threads_started":1}
I20260926 21:26:55.229224 30735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8a5c18fe2343299822e8797f999b63" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" is_pre_election: true
I20260926 21:26:55.229790 30735 raft_consensus.cc:2454] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 0.
I20260926 21:26:55.229827 30601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8a5c18fe2343299822e8797f999b63" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195ea3d896c04154a6a797e4ba5c4da7" is_pre_election: true
I20260926 21:26:55.230384 30601 raft_consensus.cc:2454] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 0.
I20260926 21:26:55.230711 30804 leader_election.cc:302] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb; no voters:
I20260926 21:26:55.231201 30939 raft_consensus.cc:2781] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:26:55.231379 30939 raft_consensus.cc:482] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:26:55.231513 30939 raft_consensus.cc:3037] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:55.234162 30939 raft_consensus.cc:504] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.234954 30939 leader_election.cc:288] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 election: Requested vote from peers 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271), 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011)
I20260926 21:26:55.235605 30735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8a5c18fe2343299822e8797f999b63" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
I20260926 21:26:55.235607 30601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad8a5c18fe2343299822e8797f999b63" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195ea3d896c04154a6a797e4ba5c4da7"
I20260926 21:26:55.235862 30735 raft_consensus.cc:3037] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:55.235888 30601 raft_consensus.cc:3037] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:26:55.238762 30601 raft_consensus.cc:2454] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 1.
I20260926 21:26:55.239043 30735 raft_consensus.cc:2454] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 1.
I20260926 21:26:55.239470 30802 leader_election.cc:302] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0604685c49f84fbca980887373eafd3e, 195ea3d896c04154a6a797e4ba5c4da7; no voters:
I20260926 21:26:55.240411 30939 raft_consensus.cc:2781] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:26:55.241050 30939 raft_consensus.cc:686] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 1 LEADER]: Becoming Leader. State: Replica: 0604685c49f84fbca980887373eafd3e, State: Running, Role: LEADER
I20260926 21:26:55.241865 30939 consensus_queue.cc:237] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } }
I20260926 21:26:55.247141 30456 catalog_manager.cc:5495] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0604685c49f84fbca980887373eafd3e (127.14.142.3). New cstate: current_term: 1 leader_uuid: "0604685c49f84fbca980887373eafd3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } health_report { overall_health: UNKNOWN } } }
I20260926 21:26:55.268623 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.262203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.14.142.3:34999)
0926 21:26:55.262412 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.262417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.262437 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:26:55.262624 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.262629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.262721 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.262856 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:26:55.263317 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.264431 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.266192 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.266739 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.266854 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.267405 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:26:55.267953 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:26:55.268068 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.268227 (+ 159us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.268302 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 21:26:55.277349 30735 raft_consensus.cc:1261] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Refusing update from remote peer 0604685c49f84fbca980887373eafd3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:55.278049 30601 raft_consensus.cc:1261] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Refusing update from remote peer 0604685c49f84fbca980887373eafd3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:26:55.278512 30943 consensus_queue.cc:1035] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:55.279021 30939 consensus_queue.cc:1035] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:26:55.297120 30948 mvcc.cc:205] Tried to move back new op lower bound from 7333716030564630528 to 7333716030434721792. Current Snapshot: MvccSnapshot[applied={T|T < 7333716030564630528}]
I20260926 21:26:55.313426 30457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:55.313853 30457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:26:55.318643 30457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Sending DeleteTablet for 3 replicas of tablet ad8a5c18fe2343299822e8797f999b63
I20260926 21:26:55.320010 30849 tablet_service.cc:1514] Processing DeleteTablet for tablet ad8a5c18fe2343299822e8797f999b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:55 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:56728
I20260926 21:26:55.319998 30581 tablet_service.cc:1514] Processing DeleteTablet for tablet ad8a5c18fe2343299822e8797f999b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:55 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:46706
I20260926 21:26:55.320622 30715 tablet_service.cc:1514] Processing DeleteTablet for tablet ad8a5c18fe2343299822e8797f999b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:26:55 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:37994
I20260926 21:26:55.321537 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30518
I20260926 21:26:55.337255 30957 tablet_replica.cc:330] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: stopping tablet replica
I20260926 21:26:55.337949 30957 raft_consensus.cc:2229] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:55.338594 30957 raft_consensus.cc:2258] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:55.339807 30956 tablet_replica.cc:330] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: stopping tablet replica
I20260926 21:26:55.340261 30956 raft_consensus.cc:2229] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:26:55.341043 30956 raft_consensus.cc:2258] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:55.341074 30957 ts_tablet_manager.cc:1905] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:55.342211 30955 tablet_replica.cc:330] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: stopping tablet replica
I20260926 21:26:55.342667 30955 raft_consensus.cc:2229] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:26:55.343161 30955 raft_consensus.cc:2258] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:26:55.343981 30956 ts_tablet_manager.cc:1905] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:55.349509 30957 ts_tablet_manager.cc:1918] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:55.349711 30957 log.cc:1196] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/ad8a5c18fe2343299822e8797f999b63
I20260926 21:26:55.350266 30957 ts_tablet_manager.cc:1939] T ad8a5c18fe2343299822e8797f999b63 P 9a97f6079b3e4d3a869a78e4d857d6cb: Deleting consensus metadata
I20260926 21:26:55.352366 30443 catalog_manager.cc:4841] TS 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): tablet ad8a5c18fe2343299822e8797f999b63 (table test-table [id=0ed7f5d74d5848dc890768f977c5dbd7]) successfully deleted
Sep 26 21:26:55 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:26:55.353686 30956 ts_tablet_manager.cc:1918] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:55.353896 30956 log.cc:1196] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/ad8a5c18fe2343299822e8797f999b63
I20260926 21:26:55.354212 30955 ts_tablet_manager.cc:1905] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:26:55.354516 30956 ts_tablet_manager.cc:1939] T ad8a5c18fe2343299822e8797f999b63 P 0604685c49f84fbca980887373eafd3e: Deleting consensus metadata
I20260926 21:26:55.356678 30444 catalog_manager.cc:4841] TS 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): tablet ad8a5c18fe2343299822e8797f999b63 (table test-table [id=0ed7f5d74d5848dc890768f977c5dbd7]) successfully deleted
I20260926 21:26:55.362095 30955 ts_tablet_manager.cc:1918] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:26:55.362267 30955 log.cc:1196] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/ad8a5c18fe2343299822e8797f999b63
I20260926 21:26:55.362766 30955 ts_tablet_manager.cc:1939] T ad8a5c18fe2343299822e8797f999b63 P 195ea3d896c04154a6a797e4ba5c4da7: Deleting consensus metadata
I20260926 21:26:55.364725 30441 catalog_manager.cc:4841] TS 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271): tablet ad8a5c18fe2343299822e8797f999b63 (table test-table [id=0ed7f5d74d5848dc890768f977c5dbd7]) successfully deleted
I20260926 21:26:55.365952 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.322579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.14.142.1:46271)
0926 21:26:55.322827 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.322836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.338904 (+ 16068us) server_negotiation.cc:407] Connection header received
0926 21:26:55.339135 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.339143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.339285 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.339432 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:55.342187 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.343405 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.346168 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.346937 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.347062 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.354905 (+ 7843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:55.354934 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:55.354956 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:55.355011 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:55.358788 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:55.359791 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:55.359798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:55.359800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:55.359953 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:55.360491 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:55.360495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:55.360497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:55.361101 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:55.361443 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.365329 (+ 3886us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.365578 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 21:26:55.366612 30641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:55.461742 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30518
I20260926 21:26:55.481303 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30652
Sep 26 21:26:55 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:26:55.501256 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.482235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.14.142.2:33011)
0926 21:26:55.482433 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.482440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.482460 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:26:55.482561 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.482566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.482661 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.482775 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:55.484082 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.485301 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.486991 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.487630 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.487774 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.493454 (+ 5680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:55.493476 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:55.493498 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:55.493565 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:55.497653 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:55.498528 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:55.498538 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:55.498542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:55.498704 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:55.499080 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:55.499085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:55.499090 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:55.499646 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:55.499946 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.500615 (+ 669us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.500861 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 21:26:55.502252 30775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:55.596931 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30652
I20260926 21:26:55.615197 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30786
Sep 26 21:26:55 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:26:55.635943 30929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:26:55.616044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.14.142.3:34999)
0926 21:26:55.616222 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 21:26:55.616230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:26:55.616302 (+ 72us) server_negotiation.cc:407] Connection header received
0926 21:26:55.616517 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:26:55.616522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:26:55.616651 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:26:55.617024 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:26:55.619007 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.619877 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.621747 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:26:55.622234 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:26:55.622578 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:26:55.627802 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:26:55.627818 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:26:55.627839 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:26:55.627910 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:26:55.632414 (+ 4504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:55.633495 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:55.633506 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:55.633518 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:55.633710 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:26:55.634147 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:26:55.634152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:26:55.634156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:26:55.634676 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:26:55.635005 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0926 21:26:55.635264 (+ 259us) server_negotiation.cc:299] Negotiation successful
0926 21:26:55.635535 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:26:55.636870 30909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:55.759672 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30786
I20260926 21:26:55.780879 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30425
I20260926 21:26:55.782140 30487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:26:55.900492 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30425
I20260926 21:27:15.936599 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36153
--webserver_interface=127.14.142.62
--webserver_port=45111
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:27:16.307536 30963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:16.307847 30963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:16.308017 30963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:16.315688 30963 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 21:27:16.315809 30963 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 21:27:16.315850 30963 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 21:27:16.315932 30963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:27:16.316030 30963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:16.316119 30963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:16.316179 30963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:27:16.316282 30963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:27:16.326840 30963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:36153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=45111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:16.328244 30963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:16.329634 30963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:16.360916 30970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:16.360961 30969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:16.360994 30972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:16.362192 30963 server_base.cc:1034] running on GCE node
I20260926 21:27:16.363268 30963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:16.365198 30963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:16.366520 30963 hybrid_clock.cc:648] HybridClock initialized: now 1790458036366438 us; error 51 us; skew 500 ppm
I20260926 21:27:16.366979 30963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:16 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458036, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:16.371132 30963 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:27:16.373164 30963 webserver.cc:460] Webserver started at http://127.14.142.62:45111/ using document root <none> and password file <none>
I20260926 21:27:16.374027 30963 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:16.374195 30963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:16.382860 30963 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 21:27:16.386838 30979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:16.388811 30963 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:27:16.389036 30963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "9f708d0b859244e3b2c8b67004fe8fb2"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "f40046ef5f30ebf82d675d43417b6d2b"
server_key_iv: "618a45f0e8113446b54db1930548ed43"
server_key_version: "encryptionkey@0"
I20260926 21:27:16.390223 30963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:16.406203 30963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:16.411044 30963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:16.411368 30963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:16.433338 30963 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:36153
I20260926 21:27:16.433398 31030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:36153 every 8 connection(s)
I20260926 21:27:16.435003 30963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:27:16.436699 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 30963
I20260926 21:27:16.437842 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:46271
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:36153
--webserver_port=35405
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:27:16.443231 31031 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:27:16.446820 31031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap starting.
I20260926 21:27:16.468972 31031 log.cc:824] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:16.481288 31031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:27:16.481879 31031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap complete.
I20260926 21:27:16.492111 31031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:16.493487 31031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f708d0b859244e3b2c8b67004fe8fb2, State: Initialized, Role: FOLLOWER
I20260926 21:27:16.494406 31031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:16.494803 31031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:27:16.494966 31031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:27:16.495231 31031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 1 FOLLOWER]: Advancing to term 2
I20260926 21:27:16.500705 31031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:16.501499 31031 leader_election.cc:302] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f708d0b859244e3b2c8b67004fe8fb2; no voters:
I20260926 21:27:16.502553 31031 leader_election.cc:288] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 21:27:16.502723 31035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 21:27:16.504325 31035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 LEADER]: Becoming Leader. State: Replica: 9f708d0b859244e3b2c8b67004fe8fb2, State: Running, Role: LEADER
I20260926 21:27:16.505028 31035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:16.505419 31031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:27:16.507609 31037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f708d0b859244e3b2c8b67004fe8fb2. Latest consensus state: current_term: 2 leader_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } } }
I20260926 21:27:16.508005 31037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:27:16.508682 31036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } } }
I20260926 21:27:16.508991 31036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:27:16.519627 31042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:27:16.528405 31042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:27:16.530290 31042 catalog_manager.cc:1259] Loaded cluster ID: 78d59a37a4664c94a946612dbf5f81c9
I20260926 21:27:16.530431 31042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:27:16.535409 31042 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:27:16.538034 31042 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Loaded TSK: 0
I20260926 21:27:16.582844 31042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Generated new TSK 1
I20260926 21:27:16.583302 31042 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 21:27:16.831288 31033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:16.831660 31033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:16.831828 31033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:16.839591 31033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:16.839781 31033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:16.839963 31033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:27:16.850291 31033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:46271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=35405
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:16.851672 31033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:16.853058 31033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:16.869454 31059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:16.869583 31058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:16.869805 31061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:16.871050 31033 server_base.cc:1034] running on GCE node
I20260926 21:27:16.871791 31033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:16.872972 31033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:16.874250 31033 hybrid_clock.cc:648] HybridClock initialized: now 1790458036874181 us; error 55 us; skew 500 ppm
I20260926 21:27:16.874636 31033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:16 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458036, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:16.878350 31033 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:27:16.880129 31033 webserver.cc:460] Webserver started at http://127.14.142.1:35405/ using document root <none> and password file <none>
I20260926 21:27:16.880946 31033 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:16.881114 31033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:16.887179 31033 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:27:16.890494 31068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:16.892125 31033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260926 21:27:16.892335 31033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "195ea3d896c04154a6a797e4ba5c4da7"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "cb78809b5ab0ac881ce8b191c0159590"
server_key_iv: "cb969eb0dfce71c62d354a81001f044d"
server_key_version: "encryptionkey@0"
I20260926 21:27:16.893560 31033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:16.911597 31033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:16.916823 31033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:16.917145 31033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:16.918382 31033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:16.920014 31033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:16.920169 31033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:16.920307 31033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:16.920410 31033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:16.962720 31033 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46271
I20260926 21:27:16.962776 31180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46271 every 8 connection(s)
I20260926 21:27:16.964959 31033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:27:16.972393 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31033
I20260926 21:27:16.973227 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:33011
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:36153
--webserver_port=39213
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:27:16 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458036, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:27:17.000231 31184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:16.969627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:37005)
0926 21:27:16.970844 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:16.970907 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:16.973522 (+ 2615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:16.975652 (+ 2130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:16.975669 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:16.976592 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:16.977512 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:16.977543 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:16.979642 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:16.979650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:16.981895 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:16.981912 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:16.983783 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:16.983796 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:16.984074 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:16.985129 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:16.985164 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:16.987898 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:16.994152 (+ 6254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:16.994161 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:16.994175 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:16.994877 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:16.995513 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:16.995517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:16.995520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:16.995755 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:16.997069 (+ 1314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:16.997085 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:16.997725 (+ 640us) client_negotiation.cc:769] Sending connection context
0926 21:27:16.998086 (+ 361us) client_negotiation.cc:240] Negotiation successful
0926 21:27:16.998839 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":464,"threads_started":1}
I20260926 21:27:17.000859 31183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:16.969085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:37005 (local address 127.14.142.62:36153)
0926 21:27:16.970021 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:16.970035 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:16.971110 (+ 1075us) server_negotiation.cc:407] Connection header received
0926 21:27:16.974255 (+ 3145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:16.974269 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:16.974908 (+ 639us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:16.975452 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:16.977881 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:16.979387 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:16.982149 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:16.983408 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:16.983668 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:16.988290 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:16.988304 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:16.988328 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:16.988392 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:16.993852 (+ 5460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:16.995082 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:16.995096 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:16.995106 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:16.995365 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:16.995895 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:16.995903 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:16.995908 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:16.996738 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:16.996982 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:16.998457 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0926 21:27:16.999101 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":562,"threads_started":1}
I20260926 21:27:17.004421 31181 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36153
I20260926 21:27:17.004856 31181 heartbeater.cc:464] Registering TS with master...
I20260926 21:27:17.005965 31181 heartbeater.cc:511] Master 127.14.142.62:36153 requested a full tablet report, sending...
I20260926 21:27:17.009459 30996 ts_manager.cc:194] Registered new tserver with Master: 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271)
I20260926 21:27:17.011345 30996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:37005
W20260926 21:27:17.349942 31185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:17.350298 31185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:17.350468 31185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:17.358085 31185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:17.358250 31185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:17.358433 31185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:27:17.368721 31185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:33011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=39213
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:17.370096 31185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:17.371455 31185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:17.385075 31190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:17.385169 31191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:17.385206 31193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:17.385854 31185 server_base.cc:1034] running on GCE node
I20260926 21:27:17.386482 31185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:17.387470 31185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:17.388671 31185 hybrid_clock.cc:648] HybridClock initialized: now 1790458037388623 us; error 52 us; skew 500 ppm
I20260926 21:27:17.389014 31185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:17 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:17.392601 31185 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:27:17.394769 31185 webserver.cc:460] Webserver started at http://127.14.142.2:39213/ using document root <none> and password file <none>
I20260926 21:27:17.395668 31185 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:17.395848 31185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:17.402410 31185 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:27:17.405843 31200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:17.407938 31185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:27:17.408130 31185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "202c9c8dd816ef4a137be667f187c309"
server_key_iv: "917a0f54a8a5a0df5779ec783aae4a05"
server_key_version: "encryptionkey@0"
I20260926 21:27:17.409268 31185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:17.426477 31185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:17.431275 31185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:17.431564 31185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:17.432682 31185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:17.434588 31185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:17.434711 31185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:17.434819 31185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:17.434909 31185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:17.474424 31185 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33011
I20260926 21:27:17.474473 31312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33011 every 8 connection(s)
I20260926 21:27:17.476065 31185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:27:17.485940 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31185
I20260926 21:27:17.486737 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:34999
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:36153
--webserver_port=32951
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:27:17 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:27:17.503507 31183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:17.479077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:55229 (local address 127.14.142.62:36153)
0926 21:27:17.479311 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:17.479315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:17.480574 (+ 1259us) server_negotiation.cc:407] Connection header received
0926 21:27:17.482506 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:17.482512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:17.482629 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:17.482743 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:17.484343 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:17.485183 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:17.487775 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:17.488898 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:17.489127 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:17.495142 (+ 6015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:17.495147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:17.495151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:17.495190 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:17.499211 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:17.500415 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:17.500422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:17.500424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:17.500681 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:17.501261 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:17.501268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:17.501273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:17.501645 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:17.501789 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:17.502914 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0926 21:27:17.503194 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 21:27:17.505290 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:17.479580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:55229)
0926 21:27:17.480361 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:17.480400 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:17.482208 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:17.482989 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:17.482998 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:17.483540 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:17.484158 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:17.484177 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:17.485362 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:17.485366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:17.487530 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:17.487550 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:17.489526 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:17.489537 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:17.489967 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:17.491084 (+ 1117us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:17.491117 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:17.494817 (+ 3700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:17.499452 (+ 4635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:17.499465 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:17.499490 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:17.500230 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:17.500826 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:17.500833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:17.500840 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:17.501114 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:17.501811 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:17.501824 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:17.502355 (+ 531us) client_negotiation.cc:769] Sending connection context
0926 21:27:17.502689 (+ 334us) client_negotiation.cc:240] Negotiation successful
0926 21:27:17.503337 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":316,"threads_started":1}
I20260926 21:27:17.507875 31313 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36153
I20260926 21:27:17.508246 31313 heartbeater.cc:464] Registering TS with master...
I20260926 21:27:17.509097 31313 heartbeater.cc:511] Master 127.14.142.62:36153 requested a full tablet report, sending...
I20260926 21:27:17.510859 30996 ts_manager.cc:194] Registered new tserver with Master: 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011)
I20260926 21:27:17.512333 30996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:55229
W20260926 21:27:17.858610 31316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:17.858917 31316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:17.859105 31316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:17.866888 31316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:17.866994 31316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:17.867188 31316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:27:17.877197 31316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:34999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=32951
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:17.878544 31316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:17.879855 31316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:17.893702 31322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:17.893615 31324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:17.893620 31321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:17.894950 31316 server_base.cc:1034] running on GCE node
I20260926 21:27:17.896302 31316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:17.897481 31316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:17.898789 31316 hybrid_clock.cc:648] HybridClock initialized: now 1790458037898719 us; error 48 us; skew 500 ppm
I20260926 21:27:17.899111 31316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:17 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:17.903118 31316 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:27:17.905428 31316 webserver.cc:460] Webserver started at http://127.14.142.3:32951/ using document root <none> and password file <none>
I20260926 21:27:17.906198 31316 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:17.906368 31316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:17.912092 31316 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:27:17.915279 31331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:17.916989 31316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 21:27:17.917205 31316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "0604685c49f84fbca980887373eafd3e"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "b5147f55b79929b5f1f892a11220d920"
server_key_iv: "12ce50e119244b9724c35ca202b9f8ce"
server_key_version: "encryptionkey@0"
I20260926 21:27:17.918215 31316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:17.956303 31316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:17.960815 31316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:17.961139 31316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:17.962337 31316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:17.963752 31316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:17.963841 31316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:17.963989 31316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:17.964058 31316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:18.002373 31316 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:34999
I20260926 21:27:18.002465 31443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:34999 every 8 connection(s)
I20260926 21:27:18.003937 31316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:27:18.013851 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31316
I20260926 21:27:18.015149 31181 heartbeater.cc:503] Master 127.14.142.62:36153 was elected leader, sending a full tablet report...
Sep 26 21:27:18 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458037, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:27:18.028277 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.006802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:50331 (local address 127.14.142.62:36153)
0926 21:27:18.007440 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.007446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.008414 (+ 968us) server_negotiation.cc:407] Connection header received
0926 21:27:18.010510 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.010517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.010620 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.010750 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:18.012345 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.013281 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.016118 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.016909 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.017064 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.021480 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:18.021486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:18.021489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:18.021523 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:18.025084 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.025877 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.025884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.025890 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.026015 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.026453 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.026459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.026463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.026729 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:18.026876 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.027610 (+ 734us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.027832 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":387,"threads_started":1}
I20260926 21:27:18.029383 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.007309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:50331)
0926 21:27:18.008189 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.008233 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:18.010200 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:18.011013 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:18.011022 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:18.011574 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:18.012152 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.012169 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.013469 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.013476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.015928 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.015943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.017248 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.017256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.017586 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.018495 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:18.018525 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:18.021244 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:18.025249 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:18.025259 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:18.025285 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:18.025749 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:18.026148 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:18.026153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:18.026155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:18.026337 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:18.026864 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:18.026873 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:18.027238 (+ 365us) client_negotiation.cc:769] Sending connection context
0926 21:27:18.027513 (+ 275us) client_negotiation.cc:240] Negotiation successful
0926 21:27:18.028063 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":388,"threads_started":1}
I20260926 21:27:18.030848 31444 heartbeater.cc:347] Connected to a master server at 127.14.142.62:36153
I20260926 21:27:18.031165 31444 heartbeater.cc:464] Registering TS with master...
I20260926 21:27:18.031908 31444 heartbeater.cc:511] Master 127.14.142.62:36153 requested a full tablet report, sending...
I20260926 21:27:18.033515 30996 ts_manager.cc:194] Registered new tserver with Master: 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
I20260926 21:27:18.033756 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.015197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.14.142.62:36153)
0926 21:27:18.016263 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.016273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.017834 (+ 1561us) server_negotiation.cc:407] Connection header received
0926 21:27:18.017970 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.017978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.018074 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.018169 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:18.020096 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.021095 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.022821 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.023336 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.023496 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.026935 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:18.026941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:18.026944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:18.026983 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:18.030527 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.031391 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.031398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.031405 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.031522 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.031915 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.031921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.031924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.032252 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:18.032509 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.032875 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.033116 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":596,"threads_started":1}
I20260926 21:27:18.034535 30996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:50331
I20260926 21:27:18.036506 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:27:18.044060 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.037646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.14.142.62:36153)
0926 21:27:18.037853 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.037862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.038253 (+ 391us) server_negotiation.cc:407] Connection header received
0926 21:27:18.038510 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.038514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.038650 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.038807 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:27:18.039289 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.040055 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.041990 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.042467 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.042608 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.043135 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:27:18.043353 (+ 218us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 21:27:18.043740 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47996: token expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 21:27:18.044230 31448 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47996: token expired
W20260926 21:27:18.044813 30919 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.14.142.62:36153)
I20260926 21:27:18.059221 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.045336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.14.142.62:36153)
0926 21:27:18.045512 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.045516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.045531 (+ 15us) server_negotiation.cc:407] Connection header received
0926 21:27:18.045656 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.045659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.045713 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.045792 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:18.047499 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.048426 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.050117 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.050690 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.050801 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.053784 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:18.053789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:18.053793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:18.053828 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:18.056470 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.057257 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.057261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.057264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.057414 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.057850 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.057854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.057856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.058205 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:18.058420 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.058709 (+ 289us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.058884 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:27:18.064162 30996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:27:18.072132 30996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:27:18.108232 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.086484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37702 (local address 127.14.142.3:34999)
0926 21:27:18.097717 (+ 11233us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.097730 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.099991 (+ 2261us) server_negotiation.cc:407] Connection header received
0926 21:27:18.100094 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.100100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.100291 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.100457 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:27:18.101337 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.103343 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.105431 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.106871 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.107003 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.107220 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.107711 (+ 491us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.107875 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10978,"thread_start_us":10812,"threads_started":1}
I20260926 21:27:18.108311 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.085696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34999 (local address 127.0.0.1:37702)
0926 21:27:18.099797 (+ 14101us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.099815 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:18.099929 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:18.100674 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:18.100684 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:18.100719 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:27:18.101165 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.101177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.103486 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.103491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.105263 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.105276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.107340 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.107344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.107480 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.107511 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 21:27:18.107651 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 21:27:18.107849 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13992,"mutex_wait_us":2,"spinlock_wait_cycles":4992,"thread_start_us":10595,"threads_started":1}
I20260926 21:27:18.109808 31456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.085099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33011 (local address 127.0.0.1:52098)
0926 21:27:18.098714 (+ 13615us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.098740 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:18.098910 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:18.100752 (+ 1842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:18.100757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:18.100801 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:27:18.101491 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.101506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.103227 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.103232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.106042 (+ 2810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.106055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.108935 (+ 2880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.108942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.109145 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.109175 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 21:27:18.109250 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 21:27:18.109374 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13440,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":13269,"threads_started":1}
I20260926 21:27:18.109835 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.085204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54408 (local address 127.14.142.1:46271)
0926 21:27:18.099508 (+ 14304us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.099519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.099567 (+ 48us) server_negotiation.cc:407] Connection header received
0926 21:27:18.099678 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.099689 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.099979 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.100214 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:27:18.101050 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.102585 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.106269 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.108048 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.108226 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.108498 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.109229 (+ 731us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.109437 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14124,"thread_start_us":13985,"threads_started":1}
I20260926 21:27:18.109869 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.084880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54408)
0926 21:27:18.095670 (+ 10790us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.095749 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:18.096038 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:18.100219 (+ 4181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:18.100226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:18.100255 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:27:18.100800 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.100813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.102740 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.102744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.106000 (+ 3256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.106020 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.108721 (+ 2701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.108729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.109003 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.109031 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 21:27:18.109156 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 21:27:18.109311 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10383,"mutex_wait_us":57,"thread_start_us":14669,"threads_started":1}
I20260926 21:27:18.110813 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.085713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.14.142.2:33011)
0926 21:27:18.099894 (+ 14181us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.099902 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.099932 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:27:18.100035 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.100042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.100250 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.100453 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:27:18.101668 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.103107 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.106747 (+ 3640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.108418 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.108621 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.108841 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.110171 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.110433 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12306,"thread_start_us":12192,"threads_started":1}
I20260926 21:27:18.112847 31379 tablet_service.cc:1467] Processing CreateTablet for tablet ea4c388f0d284e5da4b400e7d3b44a13 (DEFAULT_TABLE table=test-table [id=27fe9ce5d7734876986624cfcba9da5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:27:18.114418 31379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4c388f0d284e5da4b400e7d3b44a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:27:18.114163 31116 tablet_service.cc:1467] Processing CreateTablet for tablet ea4c388f0d284e5da4b400e7d3b44a13 (DEFAULT_TABLE table=test-table [id=27fe9ce5d7734876986624cfcba9da5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:27:18.114883 31248 tablet_service.cc:1467] Processing CreateTablet for tablet ea4c388f0d284e5da4b400e7d3b44a13 (DEFAULT_TABLE table=test-table [id=27fe9ce5d7734876986624cfcba9da5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:27:18.115513 31116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4c388f0d284e5da4b400e7d3b44a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:27:18.116108 31248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea4c388f0d284e5da4b400e7d3b44a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:27:18.130008 31461 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Bootstrap starting.
I20260926 21:27:18.132418 31462 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Bootstrap starting.
I20260926 21:27:18.136711 31461 tablet_bootstrap.cc:654] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Neither blocks nor log segments found. Creating new log.
I20260926 21:27:18.137472 31463 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Bootstrap starting.
I20260926 21:27:18.137710 31462 tablet_bootstrap.cc:654] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Neither blocks nor log segments found. Creating new log.
I20260926 21:27:18.138301 31461 log.cc:824] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:18.138897 31462 log.cc:824] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:18.142100 31463 tablet_bootstrap.cc:654] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Neither blocks nor log segments found. Creating new log.
I20260926 21:27:18.142253 31461 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: No bootstrap required, opened a new log
I20260926 21:27:18.142514 31461 ts_tablet_manager.cc:1397] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260926 21:27:18.143627 31463 log.cc:824] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:18.144441 31462 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: No bootstrap required, opened a new log
I20260926 21:27:18.144778 31462 ts_tablet_manager.cc:1397] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260926 21:27:18.147078 31463 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: No bootstrap required, opened a new log
I20260926 21:27:18.147415 31463 ts_tablet_manager.cc:1397] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260926 21:27:18.150051 31461 raft_consensus.cc:348] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.150439 31461 raft_consensus.cc:374] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:27:18.150549 31461 raft_consensus.cc:729] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195ea3d896c04154a6a797e4ba5c4da7, State: Initialized, Role: FOLLOWER
I20260926 21:27:18.151211 31461 consensus_queue.cc:260] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.154021 31461 ts_tablet_manager.cc:1428] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Time spent starting tablet: real 0.011s user 0.010s sys 0.002s
I20260926 21:27:18.156816 31462 raft_consensus.cc:348] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.157271 31462 raft_consensus.cc:374] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:27:18.157420 31462 raft_consensus.cc:729] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0604685c49f84fbca980887373eafd3e, State: Initialized, Role: FOLLOWER
I20260926 21:27:18.158161 31462 consensus_queue.cc:260] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.159693 31444 heartbeater.cc:503] Master 127.14.142.62:36153 was elected leader, sending a full tablet report...
I20260926 21:27:18.159706 31463 raft_consensus.cc:348] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.160184 31463 raft_consensus.cc:374] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:27:18.160279 31463 raft_consensus.cc:729] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a97f6079b3e4d3a869a78e4d857d6cb, State: Initialized, Role: FOLLOWER
I20260926 21:27:18.160591 31462 ts_tablet_manager.cc:1428] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Time spent starting tablet: real 0.016s user 0.009s sys 0.005s
I20260926 21:27:18.160869 31463 consensus_queue.cc:260] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.162140 31313 heartbeater.cc:503] Master 127.14.142.62:36153 was elected leader, sending a full tablet report...
I20260926 21:27:18.162665 31463 ts_tablet_manager.cc:1428] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260926 21:27:18.217923 31182 tablet.cc:2369] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:27:18.228412 31314 tablet.cc:2369] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:27:18.254870 31468 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:27:18.255242 31468 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
W20260926 21:27:18.255693 31445 tablet.cc:2369] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:27:18.257773 31468 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271), 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011)
I20260926 21:27:18.264938 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.257890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:47369 (local address 127.14.142.1:46271)
0926 21:27:18.258103 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.258111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.258145 (+ 34us) server_negotiation.cc:407] Connection header received
0926 21:27:18.258331 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.258338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.258426 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.258551 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:27:18.259198 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.260273 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.262710 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.263963 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.264131 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.264227 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.264501 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.264667 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 21:27:18.264938 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.257654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.14.142.3:47369)
0926 21:27:18.257951 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.257978 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:18.258130 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:18.258782 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:18.258787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:18.258818 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:27:18.259016 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.259025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.260547 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.260594 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.262500 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.262518 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.264133 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.264139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.264340 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.264366 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 21:27:18.264476 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 21:27:18.264667 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260926 21:27:18.266139 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.258406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48759 (local address 127.14.142.2:33011)
0926 21:27:18.258663 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.258670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.259130 (+ 460us) server_negotiation.cc:407] Connection header received
0926 21:27:18.259322 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.259326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.259394 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.259480 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:27:18.260116 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.261561 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.263999 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.264978 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.265191 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.265257 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.265856 (+ 599us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.265963 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 21:27:18.266073 31136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea4c388f0d284e5da4b400e7d3b44a13" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195ea3d896c04154a6a797e4ba5c4da7" is_pre_election: true
I20260926 21:27:18.266371 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.257684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33011 (local address 127.14.142.3:48759)
0926 21:27:18.259012 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.259026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:18.259176 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:18.259528 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:18.259534 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:18.259561 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:27:18.259927 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.259938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.261784 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.261789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.263770 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:18.263787 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.265392 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.265397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:18.265687 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.265723 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:27:18.265826 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 21:27:18.265963 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1224,"mutex_wait_us":353,"thread_start_us":615,"threads_started":1}
I20260926 21:27:18.266743 31136 raft_consensus.cc:2454] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 0.
I20260926 21:27:18.267345 31268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea4c388f0d284e5da4b400e7d3b44a13" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" is_pre_election: true
I20260926 21:27:18.267490 31332 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0604685c49f84fbca980887373eafd3e, 195ea3d896c04154a6a797e4ba5c4da7; no voters:
I20260926 21:27:18.267812 31268 raft_consensus.cc:2454] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 0.
I20260926 21:27:18.267969 31468 raft_consensus.cc:2781] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:27:18.268133 31468 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:27:18.268271 31468 raft_consensus.cc:3037] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:27:18.270529 31468 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.271217 31468 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 election: Requested vote from peers 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271), 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011)
I20260926 21:27:18.271699 31136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea4c388f0d284e5da4b400e7d3b44a13" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "195ea3d896c04154a6a797e4ba5c4da7"
I20260926 21:27:18.271713 31268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea4c388f0d284e5da4b400e7d3b44a13" candidate_uuid: "0604685c49f84fbca980887373eafd3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
I20260926 21:27:18.271908 31136 raft_consensus.cc:3037] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:27:18.271916 31268 raft_consensus.cc:3037] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:27:18.274155 31268 raft_consensus.cc:2454] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 1.
I20260926 21:27:18.274168 31136 raft_consensus.cc:2454] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0604685c49f84fbca980887373eafd3e in term 1.
I20260926 21:27:18.274596 31334 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb; no voters:
I20260926 21:27:18.274968 31468 raft_consensus.cc:2781] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:27:18.275506 31468 raft_consensus.cc:686] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 1 LEADER]: Becoming Leader. State: Replica: 0604685c49f84fbca980887373eafd3e, State: Running, Role: LEADER
I20260926 21:27:18.276080 31468 consensus_queue.cc:237] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:18.280889 30996 catalog_manager.cc:5495] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0604685c49f84fbca980887373eafd3e (127.14.142.3). New cstate: current_term: 1 leader_uuid: "0604685c49f84fbca980887373eafd3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } health_report { overall_health: HEALTHY } } }
I20260926 21:27:18.307488 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.301810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.14.142.3:34999)
0926 21:27:18.302118 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.302126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.302147 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:27:18.302245 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.302249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.302338 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.302495 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:27:18.302885 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.303616 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.305139 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.305589 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.305730 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.306307 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:27:18.306869 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:27:18.307028 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.307128 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.307216 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260926 21:27:18.316776 31268 raft_consensus.cc:1261] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Refusing update from remote peer 0604685c49f84fbca980887373eafd3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:27:18.316967 31136 raft_consensus.cc:1261] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Refusing update from remote peer 0604685c49f84fbca980887373eafd3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:27:18.317973 31468 consensus_queue.cc:1035] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:27:18.318351 31471 consensus_queue.cc:1035] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:27:18.330595 31473 mvcc.cc:205] Tried to move back new op lower bound from 7333716124935086080 to 7333716124783292416. Current Snapshot: MvccSnapshot[applied={T|T < 7333716124935086080}]
I20260926 21:27:18.350196 30996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:27:18.350571 30996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:27:18.354919 30996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Sending DeleteTablet for 3 replicas of tablet ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:18.356241 31116 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4c388f0d284e5da4b400e7d3b44a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:27:18 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:54408
I20260926 21:27:18.356249 31379 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4c388f0d284e5da4b400e7d3b44a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:27:18 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:37702
I20260926 21:27:18.356389 31248 tablet_service.cc:1514] Processing DeleteTablet for tablet ea4c388f0d284e5da4b400e7d3b44a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:27:18 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:52098
I20260926 21:27:18.357926 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 31033
I20260926 21:27:18.399147 31457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.359856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.14.142.1:46271)
0926 21:27:18.360085 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.360095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.360118 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:27:18.360195 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.360203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.360308 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.360441 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:18.363213 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.365233 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.368351 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.369339 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.369520 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.381220 (+ 11700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:18.381249 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:18.381278 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:18.381350 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:18.388402 (+ 7052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.390784 (+ 2382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.390799 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.390805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.391065 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.392230 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.392238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.392244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.393379 (+ 1135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:18.395755 (+ 2376us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.395985 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.396354 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:27:18.400084 31176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:18.401310 31484 tablet_replica.cc:330] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: stopping tablet replica
I20260926 21:27:18.401492 31486 tablet_replica.cc:330] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: stopping tablet replica
I20260926 21:27:18.401856 31486 raft_consensus.cc:2229] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:27:18.401862 31484 raft_consensus.cc:2229] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:27:18.402673 31486 raft_consensus.cc:2258] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:27:18.402967 31484 raft_consensus.cc:2258] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:27:18.407115 31484 ts_tablet_manager.cc:1905] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:27:18.408528 31486 ts_tablet_manager.cc:1905] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:27:18.415180 31484 ts_tablet_manager.cc:1918] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:27:18.415385 31484 log.cc:1196] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:18.416021 31484 ts_tablet_manager.cc:1939] T ea4c388f0d284e5da4b400e7d3b44a13 P 0604685c49f84fbca980887373eafd3e: Deleting consensus metadata
I20260926 21:27:18.417339 31486 ts_tablet_manager.cc:1918] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:27:18.417608 31486 log.cc:1196] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:18.418347 30983 catalog_manager.cc:4841] TS 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): tablet ea4c388f0d284e5da4b400e7d3b44a13 (table test-table [id=27fe9ce5d7734876986624cfcba9da5c]) successfully deleted
I20260926 21:27:18.418800 31486 ts_tablet_manager.cc:1939] T ea4c388f0d284e5da4b400e7d3b44a13 P 9a97f6079b3e4d3a869a78e4d857d6cb: Deleting consensus metadata
I20260926 21:27:18.421056 30982 catalog_manager.cc:4841] TS 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): tablet ea4c388f0d284e5da4b400e7d3b44a13 (table test-table [id=27fe9ce5d7734876986624cfcba9da5c]) successfully deleted
I20260926 21:27:18.513978 31485 tablet_replica.cc:330] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: stopping tablet replica
I20260926 21:27:18.514209 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 31033
W20260926 21:27:18.533692 30980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.142.1:46271 (error 108)
I20260926 21:27:18.534832 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 31185
I20260926 21:27:18.535769 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.535259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54422)
0926 21:27:18.535456 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.535594 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 21:27:18.536108 30980 catalog_manager.cc:4584] TS 195ea3d896c04154a6a797e4ba5c4da7 (127.14.142.1:46271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea4c388f0d284e5da4b400e7d3b44a13: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
I20260926 21:27:18.552405 31458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.535698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.14.142.2:33011)
0926 21:27:18.535928 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.535937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.536067 (+ 130us) server_negotiation.cc:407] Connection header received
0926 21:27:18.536213 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.536220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.536323 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.536426 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:18.538362 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.539236 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.541452 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.541869 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.541984 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.546050 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:18.546066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:18.546082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:18.546119 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:18.549255 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.550203 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.550210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.550212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.550343 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.550753 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.550756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.550758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.551150 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:18.551371 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.551848 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.552028 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:27:18.553198 31308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:18.589953 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.589252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54428)
0926 21:27:18.589606 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.589730 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260926 21:27:18.591245 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.590814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54432)
0926 21:27:18.590988 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.591094 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 21:27:18.650559 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 31185
I20260926 21:27:18.653558 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.652949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54446)
0926 21:27:18.653224 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.653347 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260926 21:27:18.654791 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.654339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54458)
0926 21:27:18.654529 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.654633 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 21:27:18.669422 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 31316
I20260926 21:27:18.686450 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.670235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.14.142.3:34999)
0926 21:27:18.670426 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:18.670432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:18.670459 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:27:18.670593 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:18.670599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:18.670692 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:18.670812 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:18.672134 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.673401 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.674755 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:18.675428 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:18.675598 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:18.678444 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:18.678460 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:18.678482 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:18.678536 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:18.682901 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.683886 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.683896 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.683899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.684118 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:18.684611 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:18.684618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:18.684623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:18.685101 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:18.685484 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:18.685755 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 21:27:18.686018 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:27:18.687253 31439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:18.735734 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.735105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54468)
0926 21:27:18.735384 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.735500 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260926 21:27:18.737141 31459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:18.736682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:46271 (local address 127.0.0.1:54472)
0926 21:27:18.736905 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:18.736995 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260926 21:27:18.815831 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 31316
I20260926 21:27:18.839943 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 30963
I20260926 21:27:18.841506 31026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:18.963454 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 30963
I20260926 21:27:38.988639 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36153
--webserver_interface=127.14.142.62
--webserver_port=45111
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:27:39.354406 31492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:39.354720 31492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:39.354883 31492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:39.362663 31492 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 21:27:39.362766 31492 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 21:27:39.362807 31492 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 21:27:39.362887 31492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:27:39.363003 31492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:39.363090 31492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:39.363131 31492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:27:39.363180 31492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:27:39.374179 31492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:36153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=45111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:39.375682 31492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:39.377231 31492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:39.390076 31497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:39.390043 31500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:39.390143 31498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:39.390883 31492 server_base.cc:1034] running on GCE node
I20260926 21:27:39.391700 31492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:39.393275 31492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:39.394542 31492 hybrid_clock.cc:648] HybridClock initialized: now 1790458059394481 us; error 31 us; skew 500 ppm
I20260926 21:27:39.395380 31492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:39 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458059, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:39.404215 31492 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:27:39.406567 31492 webserver.cc:460] Webserver started at http://127.14.142.62:45111/ using document root <none> and password file <none>
I20260926 21:27:39.407685 31492 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:39.407871 31492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:39.414438 31492 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260926 21:27:39.417894 31507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:39.419725 31492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:27:39.419972 31492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "9f708d0b859244e3b2c8b67004fe8fb2"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "f40046ef5f30ebf82d675d43417b6d2b"
server_key_iv: "618a45f0e8113446b54db1930548ed43"
server_key_version: "encryptionkey@0"
I20260926 21:27:39.421617 31492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:39.443467 31492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:39.448280 31492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:39.448683 31492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:39.471359 31492 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:36153
I20260926 21:27:39.471464 31558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:36153 every 8 connection(s)
I20260926 21:27:39.473181 31492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:27:39.477329 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31492
I20260926 21:27:39.478384 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:46271
--local_ip_for_outbound_sockets=127.14.142.1
--tserver_master_addrs=127.14.142.62:36153
--webserver_port=35405
--webserver_interface=127.14.142.1
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:27:39.482916 31559 sys_catalog.cc:263] Verifying existing consensus state
I20260926 21:27:39.487488 31559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap starting.
I20260926 21:27:39.521741 31559 log.cc:824] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:39.547470 31559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:27:39.548281 31559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Bootstrap complete.
I20260926 21:27:39.563871 31559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:39.565439 31559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f708d0b859244e3b2c8b67004fe8fb2, State: Initialized, Role: FOLLOWER
I20260926 21:27:39.566262 31559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:39.566576 31559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:27:39.566690 31559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:27:39.566887 31559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 2 FOLLOWER]: Advancing to term 3
I20260926 21:27:39.572481 31559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:39.573259 31559 leader_election.cc:302] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f708d0b859244e3b2c8b67004fe8fb2; no voters:
I20260926 21:27:39.574002 31559 leader_election.cc:288] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 21:27:39.574213 31563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 21:27:39.575307 31563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [term 3 LEADER]: Becoming Leader. State: Replica: 9f708d0b859244e3b2c8b67004fe8fb2, State: Running, Role: LEADER
I20260926 21:27:39.576085 31559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:27:39.576002 31563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } }
I20260926 21:27:39.579226 31564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } } }
I20260926 21:27:39.579260 31565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f708d0b859244e3b2c8b67004fe8fb2. Latest consensus state: current_term: 3 leader_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f708d0b859244e3b2c8b67004fe8fb2" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 36153 } } }
I20260926 21:27:39.579703 31564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:27:39.579711 31565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:27:39.580744 31569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:27:39.603652 31569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:27:39.605289 31569 catalog_manager.cc:1259] Loaded cluster ID: 78d59a37a4664c94a946612dbf5f81c9
I20260926 21:27:39.605472 31569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:27:39.610127 31569 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:27:39.612301 31569 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Loaded TSK: 0
I20260926 21:27:39.612533 31569 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f708d0b859244e3b2c8b67004fe8fb2: Loaded TSK: 1
F20260926 21:27:39.613739 31569 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe8f69d59ad google::LogMessage::Fail() at ??:0
@ 0x7fe8f69d888a google::LogMessage::SendToLog() at ??:0
@ 0x7fe8f69d54de google::LogMessage::Flush() at ??:0
@ 0x7fe8f69d6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe8fa7397d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe8fa72873e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe90c4041c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe90c403783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe90c48f2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fe90c48f0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fe90c4d95e9 std::function<>::operator()() at ??:0
@ 0x7fe90c407cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
I20260926 21:27:39.689260 31587 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fe90c405c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe90c48d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe90c48d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe8f9621345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe8f962491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe8f9624799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe8f95ffeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe8fc8d96db start_thread at ??:0
@ 0x7fe8f586f61f clone at ??:0
W20260926 21:27:39.950296 31561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:39.950613 31561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:39.950757 31561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:39.958510 31561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:39.958662 31561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:39.958789 31561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:27:39.969642 31561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:46271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=35405
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:39.971133 31561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:39.972630 31561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:39.987186 31606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:39.987242 31607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:39.987283 31609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:39.988511 31561 server_base.cc:1034] running on GCE node
I20260926 21:27:39.989657 31561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:39.991076 31561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:39.992403 31561 hybrid_clock.cc:648] HybridClock initialized: now 1790458059992343 us; error 43 us; skew 500 ppm
I20260926 21:27:39.992930 31561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:39 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458059, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:39.996325 31561 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:27:39.998212 31561 webserver.cc:460] Webserver started at http://127.14.142.1:35405/ using document root <none> and password file <none>
I20260926 21:27:39.998961 31561 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:39.999083 31561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:40.005378 31561 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260926 21:27:40.009002 31616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:40.010641 31561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260926 21:27:40.010823 31561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "195ea3d896c04154a6a797e4ba5c4da7"
format_stamp: "Formatted at 2026-09-26 21:26:53 on dist-test-slave-4d5t"
server_key: "cb78809b5ab0ac881ce8b191c0159590"
server_key_iv: "cb969eb0dfce71c62d354a81001f044d"
server_key_version: "encryptionkey@0"
I20260926 21:27:40.011942 31561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:40.053966 31561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:40.060002 31561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:40.060400 31561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:40.061653 31561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:40.063823 31623 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 21:27:40.070086 31561 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 21:27:40.070360 31561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260926 21:27:40.070591 31561 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 21:27:40.074940 31561 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 21:27:40.075091 31561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.005s user 0.003s sys 0.000s
I20260926 21:27:40.075518 31623 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Bootstrap starting.
I20260926 21:27:40.106793 31623 log.cc:824] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:40.122566 31561 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:46271
I20260926 21:27:40.122591 31730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:46271 every 8 connection(s)
I20260926 21:27:40.123979 31623 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 21:27:40.124248 31561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:27:40.124883 31623 tablet_bootstrap.cc:492] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Bootstrap complete.
I20260926 21:27:40.126557 31623 ts_tablet_manager.cc:1397] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Time spent bootstrapping tablet: real 0.051s user 0.036s sys 0.012s
I20260926 21:27:40.130263 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.127515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:52665)
0926 21:27:40.128674 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.129086 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":519,"threads_started":1}
W20260926 21:27:40.131274 31731 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:36153 (0 consecutive failures): Network error: Failed to ping master at 127.14.142.62:36153: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
I20260926 21:27:40.132401 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.131914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:37267)
0926 21:27:40.132180 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.132267 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 21:27:40.133581 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.133035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:60071)
0926 21:27:40.133197 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.133341 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 21:27:40.134001 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31561
W20260926 21:27:40.134387 31731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 21:27:40.134827 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:33011
--local_ip_for_outbound_sockets=127.14.142.2
--tserver_master_addrs=127.14.142.62:36153
--webserver_port=39213
--webserver_interface=127.14.142.2
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:27:40.136232 31623 raft_consensus.cc:348] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:40.137348 31623 raft_consensus.cc:729] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 195ea3d896c04154a6a797e4ba5c4da7, State: Initialized, Role: FOLLOWER
I20260926 21:27:40.138001 31623 consensus_queue.cc:260] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:40.139667 31623 ts_tablet_manager.cc:1428] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260926 21:27:40.140005 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.139520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:37305)
0926 21:27:40.139745 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.139848 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 21:27:40.141357 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.140722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:48739)
0926 21:27:40.141032 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.141155 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260926 21:27:40.496918 31735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:40.497264 31735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:40.497386 31735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:40.505513 31735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:40.505787 31735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:40.505968 31735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:27:40.519385 31735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:33011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=39213
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:40.520843 31735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:40.522310 31735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:40.537977 31740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:40.538012 31743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:40.538110 31741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:40.539040 31735 server_base.cc:1034] running on GCE node
I20260926 21:27:40.539783 31735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:40.540891 31735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:40.542146 31735 hybrid_clock.cc:648] HybridClock initialized: now 1790458060542094 us; error 35 us; skew 500 ppm
I20260926 21:27:40.542510 31735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:40 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458060, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:40.546481 31735 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:27:40.548138 31735 webserver.cc:460] Webserver started at http://127.14.142.2:39213/ using document root <none> and password file <none>
I20260926 21:27:40.548921 31735 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:40.549041 31735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:40.554893 31735 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:27:40.558333 31750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:40.560057 31735 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 21:27:40.560282 31735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "9a97f6079b3e4d3a869a78e4d857d6cb"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "202c9c8dd816ef4a137be667f187c309"
server_key_iv: "917a0f54a8a5a0df5779ec783aae4a05"
server_key_version: "encryptionkey@0"
I20260926 21:27:40.561439 31735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:40.580925 31735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:40.587018 31735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:40.587343 31735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:40.588523 31735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:40.590129 31735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:40.590261 31735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:40.590395 31735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:40.590499 31735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:40.630255 31735 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33011
I20260926 21:27:40.630355 31863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33011 every 8 connection(s)
I20260926 21:27:40.631865 31735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:27:40.634377 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31735
I20260926 21:27:40.635202 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:34999
--local_ip_for_outbound_sockets=127.14.142.3
--tserver_master_addrs=127.14.142.62:36153
--webserver_port=32951
--webserver_interface=127.14.142.3
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:27:40.638065 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.635534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:53341)
0926 21:27:40.636869 (+ 1335us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.637120 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":960,"thread_start_us":543,"threads_started":1}
W20260926 21:27:40.638870 31864 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:36153 (0 consecutive failures): Network error: Failed to ping master at 127.14.142.62:36153: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
I20260926 21:27:40.639894 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.639550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:57465)
0926 21:27:40.639707 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.639789 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 21:27:40.640779 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:40.640424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:50495)
0926 21:27:40.640592 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:40.640673 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 21:27:40.640987 31864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 21:27:41.000976 31867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:41.001294 31867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:41.001398 31867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:41.008952 31867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:41.009078 31867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:41.009192 31867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:27:41.019740 31867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:34999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=32951
--tserver_master_addrs=127.14.142.62:36153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:41.021162 31867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:41.022828 31867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:41.038394 31875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:41.038514 31872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:41.038491 31873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:41.039259 31867 server_base.cc:1034] running on GCE node
I20260926 21:27:41.039968 31867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:41.041139 31867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:41.042395 31867 hybrid_clock.cc:648] HybridClock initialized: now 1790458061042331 us; error 62 us; skew 500 ppm
I20260926 21:27:41.042764 31867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:41 dist-test-slave-4d5t krb5kdc[30412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458061, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:41.046414 31867 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:27:41.048928 31867 webserver.cc:460] Webserver started at http://127.14.142.3:32951/ using document root <none> and password file <none>
I20260926 21:27:41.050135 31867 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:41.050383 31867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:41.059262 31867 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 21:27:41.062507 31882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:41.064248 31867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 21:27:41.064450 31867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "0604685c49f84fbca980887373eafd3e"
format_stamp: "Formatted at 2026-09-26 21:26:54 on dist-test-slave-4d5t"
server_key: "b5147f55b79929b5f1f892a11220d920"
server_key_iv: "12ce50e119244b9724c35ca202b9f8ce"
server_key_version: "encryptionkey@0"
I20260926 21:27:41.065861 31867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:41.080971 31867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:41.085836 31867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:41.086145 31867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:41.087450 31867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:41.089779 31867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:41.089884 31867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:41.090016 31867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:41.090111 31867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:41.129594 31867 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:34999
I20260926 21:27:41.129631 31994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:34999 every 8 connection(s)
I20260926 21:27:41.131177 31867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:27:41.135339 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 31867
I20260926 21:27:41.136529 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.134422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:34623)
0926 21:27:41.135178 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.135410 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":332,"threads_started":1}
W20260926 21:27:41.137346 31995 heartbeater.cc:644] Failed to heartbeat to 127.14.142.62:36153 (0 consecutive failures): Network error: Failed to ping master at 127.14.142.62:36153: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
I20260926 21:27:41.138267 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.137889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:57585)
0926 21:27:41.138074 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.138164 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 21:27:41.139158 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.138854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:33203)
0926 21:27:41.138977 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.139052 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 21:27:41.139431 31995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 21:27:41.143129 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.142379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:51455)
0926 21:27:41.142837 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.142932 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":220,"threads_started":1}
I20260926 21:27:41.327921 32000 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:27:41.328249 32000 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:41.330020 32000 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011), 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
Sep 26 21:27:41 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458059, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
Sep 26 21:27:41 dist-test-slave-4d5t krb5kdc[30412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458059, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:27:41.356268 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.330474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33011 (local address 127.14.142.1:38721)
0926 21:27:41.332931 (+ 2457us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.332963 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:41.333121 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:41.337454 (+ 4333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:41.337461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:41.337975 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:41.338544 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:41.338566 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.340326 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.340330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:41.341997 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:41.342015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.343196 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.343203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:41.343574 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:41.344579 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:41.344636 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:41.348219 (+ 3583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:41.352135 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:41.352141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:41.352145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:41.352754 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:41.353273 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:41.353278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:41.353283 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:41.353536 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:41.354424 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:41.354434 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:41.354829 (+ 395us) client_negotiation.cc:769] Sending connection context
0926 21:27:41.355120 (+ 291us) client_negotiation.cc:240] Negotiation successful
0926 21:27:41.355803 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2306,"mutex_wait_us":234,"spinlock_wait_cycles":56320,"thread_start_us":2104,"threads_started":1}
I20260926 21:27:41.356283 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.330794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:54053 (local address 127.14.142.3:34999)
0926 21:27:41.333396 (+ 2602us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:41.333408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:41.333449 (+ 41us) server_negotiation.cc:407] Connection header received
0926 21:27:41.335673 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:41.335684 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:41.336310 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:41.336787 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:41.338815 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.340063 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.342190 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.342934 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.343183 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:41.348535 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:41.348544 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:41.348617 (+ 73us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:41.348677 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:41.351770 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:41.352967 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:41.352979 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:41.352984 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:41.353200 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:41.353658 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:41.353668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:41.353670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:41.354262 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:41.354493 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:41.355396 (+ 903us) server_negotiation.cc:299] Negotiation successful
0926 21:27:41.355874 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2406,"thread_start_us":2223,"threads_started":1}
I20260926 21:27:41.356776 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.330605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:38721 (local address 127.14.142.2:33011)
0926 21:27:41.332530 (+ 1925us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:41.332574 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:41.333272 (+ 698us) server_negotiation.cc:407] Connection header received
0926 21:27:41.335665 (+ 2393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:41.335682 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:41.336310 (+ 628us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:41.336787 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:41.338804 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.340062 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.342191 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.342934 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.343207 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:41.348612 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:41.348625 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:41.348651 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:41.348695 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:41.351960 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:41.352932 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:41.352939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:41.352945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:41.353142 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:41.353680 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:41.353684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:41.353687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:41.354263 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:41.354520 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:41.355811 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0926 21:27:41.356268 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1691,"thread_start_us":1567,"threads_started":1}
I20260926 21:27:41.357234 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.330389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34999 (local address 127.14.142.1:54053)
0926 21:27:41.330822 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.330869 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:41.331632 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:41.336971 (+ 5339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:41.336980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:41.337770 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:41.338542 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:41.338566 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.340323 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.340327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:41.342004 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:41.342018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:41.343266 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:41.343275 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:41.343578 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:41.344549 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:41.344609 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:41.348126 (+ 3517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:41.352008 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:41.352019 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:41.352038 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:41.352796 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:41.353330 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:41.353339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:41.353343 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:41.353525 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:41.354430 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:41.354435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:41.354829 (+ 394us) client_negotiation.cc:769] Sending connection context
0926 21:27:41.355120 (+ 291us) client_negotiation.cc:240] Negotiation successful
0926 21:27:41.356798 (+ 1678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":1403,"spinlock_wait_cycles":28672}
W20260926 21:27:41.361234 31619 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
W20260926 21:27:41.362403 31620 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:41.362551 31620 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 195ea3d896c04154a6a797e4ba5c4da7; no voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb
I20260926 21:27:41.362995 32000 raft_consensus.cc:2726] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 21:27:41.642542 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:41.641620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:41183)
0926 21:27:41.642118 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:41.642291 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":238,"threads_started":1}
I20260926 21:27:42.140823 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:42.139974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:51525)
0926 21:27:42.140423 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:42.140538 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
I20260926 21:27:42.144932 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:42.144009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:51435)
0926 21:27:42.144487 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:42.144747 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":264,"threads_started":1}
I20260926 21:27:42.644841 32007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:42.644032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:47465)
0926 21:27:42.644505 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:42.644668 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":261,"threads_started":1}
I20260926 21:27:43.142792 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:43.141902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:40359)
0926 21:27:43.142472 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:43.142599 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":288,"threads_started":1}
I20260926 21:27:43.146646 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:43.145869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:42241)
0926 21:27:43.146355 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:43.146451 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":216,"threads_started":1}
I20260926 21:27:43.198750 32010 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:27:43.198977 32010 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:43.199707 32010 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011), 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
W20260926 21:27:43.200862 31620 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
W20260926 21:27:43.201059 31619 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:43.201241 31619 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 195ea3d896c04154a6a797e4ba5c4da7; no voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb
I20260926 21:27:43.201567 32010 raft_consensus.cc:2726] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 21:27:43.646555 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:43.645790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:55023)
0926 21:27:43.646281 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:43.646367 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":257,"threads_started":1}
I20260926 21:27:44.144814 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:44.143876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:56449)
0926 21:27:44.144412 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:44.144513 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":288,"threads_started":1}
I20260926 21:27:44.148494 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:44.147694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:59835)
0926 21:27:44.148230 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:44.148338 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":283,"threads_started":1}
I20260926 21:27:44.648514 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:44.647741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:47873)
0926 21:27:44.648239 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:44.648330 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":239,"threads_started":1}
I20260926 21:27:45.146924 32015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:45.145955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:47783)
0926 21:27:45.146622 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:45.146732 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":411,"threads_started":1}
I20260926 21:27:45.150607 32016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:45.149642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:40285)
0926 21:27:45.150182 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:45.150408 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":321,"threads_started":1}
I20260926 21:27:45.650586 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:45.649689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:39591)
0926 21:27:45.650274 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:45.650369 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":338,"threads_started":1}
I20260926 21:27:46.148802 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:46.147917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:47713)
0926 21:27:46.148447 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:46.148545 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":292,"threads_started":1}
I20260926 21:27:46.152467 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:46.151736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:60993)
0926 21:27:46.152217 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:46.152303 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":274,"threads_started":1}
I20260926 21:27:46.210597 32020 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:27:46.210815 32020 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:46.211611 32020 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011), 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
W20260926 21:27:46.212639 31620 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
W20260926 21:27:46.212818 31619 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:46.212999 31619 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 195ea3d896c04154a6a797e4ba5c4da7; no voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb
I20260926 21:27:46.213312 32020 raft_consensus.cc:2726] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 21:27:46.653136 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:46.651947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:55209)
0926 21:27:46.652834 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:46.652943 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":459,"threads_started":1}
I20260926 21:27:47.150803 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:47.149932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:59471)
0926 21:27:47.150463 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:47.150560 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":273,"threads_started":1}
I20260926 21:27:47.154337 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:47.153533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:57533)
0926 21:27:47.154057 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:47.154154 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":291,"threads_started":1}
I20260926 21:27:47.655838 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:47.654350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:47401)
0926 21:27:47.655437 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:47.655571 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":590,"threads_started":1}
I20260926 21:27:48.152832 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:48.151906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:42351)
0926 21:27:48.152449 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:48.152542 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":258,"threads_started":1}
I20260926 21:27:48.156075 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:48.155286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:42681)
0926 21:27:48.155819 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:48.155909 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":251,"threads_started":1}
I20260926 21:27:48.658073 32027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:48.657172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:34669)
0926 21:27:48.657715 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:48.657857 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":287,"threads_started":1}
I20260926 21:27:49.154580 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:49.153831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:40403)
0926 21:27:49.154292 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:49.154384 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":242,"threads_started":1}
I20260926 21:27:49.158062 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:49.157061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:40425)
0926 21:27:49.157706 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:49.157828 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":327,"threads_started":1}
I20260926 21:27:49.660499 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:49.659430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:55763)
0926 21:27:49.660068 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:49.660204 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":301,"threads_started":1}
I20260926 21:27:50.088260 32031 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:27:50.088538 32031 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:50.089366 32031 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011), 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
W20260926 21:27:50.090633 31619 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
W20260926 21:27:50.090791 31620 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:50.090941 31620 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 195ea3d896c04154a6a797e4ba5c4da7; no voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb
I20260926 21:27:50.091249 32031 raft_consensus.cc:2726] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 21:27:50.156926 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:50.155894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:59181)
0926 21:27:50.156611 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:50.156723 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":297,"threads_started":1}
I20260926 21:27:50.160171 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:50.159257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:46953)
0926 21:27:50.159854 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:50.159958 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":329,"threads_started":1}
I20260926 21:27:50.662637 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:50.661818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:34839)
0926 21:27:50.662304 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:50.662403 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":259,"threads_started":1}
I20260926 21:27:51.158767 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:51.158032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:32931)
0926 21:27:51.158493 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:51.158588 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":232,"threads_started":1}
I20260926 21:27:51.161820 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:51.161082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:43373)
0926 21:27:51.161572 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:51.161649 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":275,"threads_started":1}
I20260926 21:27:51.664548 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:51.663682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:56939)
0926 21:27:51.664274 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:51.664372 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":299,"threads_started":1}
I20260926 21:27:52.161127 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:52.160088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:47659)
0926 21:27:52.160831 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:52.160951 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":321,"threads_started":1}
I20260926 21:27:52.163538 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:52.162813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:56641)
0926 21:27:52.163286 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:52.163368 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":263,"threads_started":1}
I20260926 21:27:52.666958 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:52.665977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:55459)
0926 21:27:52.666546 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:52.666707 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":325,"threads_started":1}
I20260926 21:27:53.162890 32041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:53.162167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:41319)
0926 21:27:53.162640 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:53.162720 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260926 21:27:53.165560 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:53.164514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:54207)
0926 21:27:53.165221 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:53.165316 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":376,"threads_started":1}
I20260926 21:27:53.668745 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:53.667875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:59407)
0926 21:27:53.668371 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:53.668468 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":267,"threads_started":1}
I20260926 21:27:54.165459 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:54.164237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:45725)
0926 21:27:54.165176 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:54.165283 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":524,"threads_started":1}
I20260926 21:27:54.167632 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:54.166681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:37827)
0926 21:27:54.167369 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:54.167450 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":408,"threads_started":1}
I20260926 21:27:54.670861 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:54.669750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:38343)
0926 21:27:54.670430 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:54.670647 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":341,"threads_started":1}
I20260926 21:27:55.068849 32047 raft_consensus.cc:482] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:27:55.069100 32047 raft_consensus.cc:504] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "195ea3d896c04154a6a797e4ba5c4da7" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 46271 } } peers { permanent_uuid: "9a97f6079b3e4d3a869a78e4d857d6cb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33011 } } peers { permanent_uuid: "0604685c49f84fbca980887373eafd3e" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34999 } }
I20260926 21:27:55.070029 32047 leader_election.cc:288] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011), 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999)
W20260926 21:27:55.071312 31620 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0604685c49f84fbca980887373eafd3e (127.14.142.3:34999): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
W20260926 21:27:55.071933 31619 leader_election.cc:341] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9a97f6079b3e4d3a869a78e4d857d6cb (127.14.142.2:33011): Not found: Tablet not found: ea4c388f0d284e5da4b400e7d3b44a13
I20260926 21:27:55.072079 31619 leader_election.cc:302] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 195ea3d896c04154a6a797e4ba5c4da7; no voters: 0604685c49f84fbca980887373eafd3e, 9a97f6079b3e4d3a869a78e4d857d6cb
I20260926 21:27:55.072388 32047 raft_consensus.cc:2726] T ea4c388f0d284e5da4b400e7d3b44a13 P 195ea3d896c04154a6a797e4ba5c4da7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 21:27:55.167318 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:55.166534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:35503)
0926 21:27:55.167031 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:55.167124 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":273,"threads_started":1}
I20260926 21:27:55.169687 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:55.168887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:51101)
0926 21:27:55.169349 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:55.169476 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":224,"threads_started":1}
I20260926 21:27:55.673199 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:55.672312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.2:46183)
0926 21:27:55.672891 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:55.673035 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":352,"threads_started":1}
I20260926 21:27:56.138777 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 31561
I20260926 21:27:56.156126 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:56.139997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.14.142.1:46271)
0926 21:27:56.140711 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:56.140724 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:56.140766 (+ 42us) server_negotiation.cc:407] Connection header received
0926 21:27:56.140860 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:56.140874 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:56.141200 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:56.141359 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:56.142914 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:56.143861 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:56.145494 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:56.146258 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:56.146519 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:56.149486 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:56.149502 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:56.149520 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:56.149563 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:56.152792 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:56.153621 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:56.153628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:56.153631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:56.153771 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:56.154305 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:56.154310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:56.154312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:56.154719 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:56.155007 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:56.155596 (+ 589us) server_negotiation.cc:299] Negotiation successful
0926 21:27:56.155827 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":861,"threads_started":1}
I20260926 21:27:56.157642 31726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:56.169581 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:56.168679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.3:41377)
0926 21:27:56.169231 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:56.169353 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":284,"threads_started":1}
I20260926 21:27:56.259641 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:56.258426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:36153 (local address 127.14.142.1:59071)
0926 21:27:56.259019 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:56.259235 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.62:36153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":299,"threads_started":1}
I20260926 21:27:56.259970 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 31561
I20260926 21:27:56.281286 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 31735
I20260926 21:27:56.297463 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:56.282250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.14.142.2:33011)
0926 21:27:56.282852 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:56.282857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:56.282874 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:27:56.282942 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:56.282946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:56.283014 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:56.283153 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:56.285461 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:56.286262 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:56.287566 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:56.288029 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:56.288172 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:56.291094 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:56.291100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:56.291103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:56.291143 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:56.294198 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:56.295058 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:56.295073 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:56.295076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:56.295220 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:56.295714 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:56.295721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:56.295724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:56.296057 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:56.296374 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:56.296747 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 21:27:56.297035 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":282,"threads_started":1}
I20260926 21:27:56.298344 31858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:56.389117 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 31735
I20260926 21:27:56.408110 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 31867
I20260926 21:27:56.423149 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:56.409096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47624 (local address 127.14.142.3:34999)
0926 21:27:56.409680 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:56.409685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:56.409702 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:27:56.409756 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:56.409759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:56.409826 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:56.409919 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:56.411410 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:56.412229 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:56.413696 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:56.414206 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:56.414322 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:56.417326 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:56.417331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:56.417335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:56.417368 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:56.420419 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:56.421370 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:56.421374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:56.421377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:56.421523 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:56.421904 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:56.421907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:56.421910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:56.422198 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:56.422447 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:56.422654 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 21:27:56.422829 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":315,"threads_started":1}
I20260926 21:27:56.424024 31990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:27:56.510694 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 31867
2026-09-26T21:27:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65269 ms)
[----------] 30 tests from SecurityITest (314515 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:27:56 dist-test-slave-4d5t krb5kdc[32061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:27:56 dist-test-slave-4d5t krb5kdc[32061](info): set up 2 sockets
Sep 26 21:27:56 dist-test-slave-4d5t krb5kdc[32061](info): commencing operation
krb5kdc: starting...
W20260926 21:27:57.665776 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:27:57 dist-test-slave-4d5t krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:27:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:27:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:27:57.961464 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44763
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:27:58.324958 32074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:58.325273 32074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:58.325434 32074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:58.332983 32074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:27:58.333101 32074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:58.333185 32074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:58.333271 32074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:27:58.333389 32074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:27:58.343524 32074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:58.345002 32074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:58.346313 32074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:58.358865 32079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:58.358886 32080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:58.359400 32082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:58.359651 32074 server_base.cc:1034] running on GCE node
I20260926 21:27:58.360615 32074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:58.362179 32074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:58.363435 32074 hybrid_clock.cc:648] HybridClock initialized: now 1790458078363387 us; error 41 us; skew 500 ppm
I20260926 21:27:58.363842 32074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:58 dist-test-slave-4d5t krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458078, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:58.367528 32074 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:27:58.369537 32074 webserver.cc:460] Webserver started at http://127.14.142.62:39609/ using document root <none> and password file <none>
I20260926 21:27:58.370321 32074 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:58.370483 32074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:58.370820 32074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:27:58.374555 32074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "94b28d7b346448abaf9d56913e41b5e8"
format_stamp: "Formatted at 2026-09-26 21:27:58 on dist-test-slave-4d5t"
server_key: "f1be5a4860cd1b52f19d1d0fb03408ee"
server_key_iv: "b3bea0d9ab21ae0151f59f883a7eabcc"
server_key_version: "encryptionkey@0"
I20260926 21:27:58.375473 32074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "94b28d7b346448abaf9d56913e41b5e8"
format_stamp: "Formatted at 2026-09-26 21:27:58 on dist-test-slave-4d5t"
server_key: "f1be5a4860cd1b52f19d1d0fb03408ee"
server_key_iv: "b3bea0d9ab21ae0151f59f883a7eabcc"
server_key_version: "encryptionkey@0"
I20260926 21:27:58.383423 32074 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260926 21:27:58.387705 32089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:58.389441 32074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:27:58.389703 32074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "94b28d7b346448abaf9d56913e41b5e8"
format_stamp: "Formatted at 2026-09-26 21:27:58 on dist-test-slave-4d5t"
server_key: "f1be5a4860cd1b52f19d1d0fb03408ee"
server_key_iv: "b3bea0d9ab21ae0151f59f883a7eabcc"
server_key_version: "encryptionkey@0"
I20260926 21:27:58.389984 32074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:58.424417 32074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:58.429533 32074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:58.429843 32074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:58.449857 32074 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:44763
I20260926 21:27:58.449857 32140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:44763 every 8 connection(s)
I20260926 21:27:58.451282 32074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:27:58.455160 32141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:27:58.460114 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 32074
I20260926 21:27:58.460350 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:27:58.461252 14904 external_mini_cluster.cc:1428] Setting key db9470624ae73178dbb737259a1e22c4
I20260926 21:27:58.468225 32141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8: Bootstrap starting.
I20260926 21:27:58.471673 32141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8: Neither blocks nor log segments found. Creating new log.
I20260926 21:27:58.472852 32141 log.cc:824] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8: Log is configured to *not* fsync() on all Append() calls
I20260926 21:27:58.476784 32141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8: No bootstrap required, opened a new log
Sep 26 21:27:58 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:27:58.487262 32141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b28d7b346448abaf9d56913e41b5e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44763 } }
I20260926 21:27:58.487732 32141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:27:58.487875 32141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94b28d7b346448abaf9d56913e41b5e8, State: Initialized, Role: FOLLOWER
I20260926 21:27:58.488628 32141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b28d7b346448abaf9d56913e41b5e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44763 } }
I20260926 21:27:58.488934 32141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:27:58.489089 32141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:27:58.489274 32141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:27:58.489575 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:58.463522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.14.142.62:44763)
0926 21:27:58.464825 (+ 1303us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:58.464845 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:58.464897 (+ 52us) server_negotiation.cc:407] Connection header received
0926 21:27:58.467139 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:58.467166 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:58.467616 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:58.468084 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:58.469990 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:58.471500 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:58.473373 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:58.474624 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:58.474992 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:58.480891 (+ 5899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:58.480908 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:58.480934 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:58.481001 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:58.484811 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:58.485639 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:58.485650 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:58.485657 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:58.485854 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:58.486289 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:58.486293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:58.486299 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:58.486903 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:58.487224 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:58.487702 (+ 478us) server_negotiation.cc:299] Negotiation successful
0926 21:27:58.488189 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":675,"threads_started":1}
I20260926 21:27:58.492254 32141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b28d7b346448abaf9d56913e41b5e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44763 } }
I20260926 21:27:58.492897 32141 leader_election.cc:302] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94b28d7b346448abaf9d56913e41b5e8; no voters:
I20260926 21:27:58.493498 32141 leader_election.cc:288] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:27:58.493892 32146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:27:58.494768 32146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [term 1 LEADER]: Becoming Leader. State: Replica: 94b28d7b346448abaf9d56913e41b5e8, State: Running, Role: LEADER
I20260926 21:27:58.495404 32146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b28d7b346448abaf9d56913e41b5e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44763 } }
I20260926 21:27:58.496150 32141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:27:58.498972 32148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94b28d7b346448abaf9d56913e41b5e8. Latest consensus state: current_term: 1 leader_uuid: "94b28d7b346448abaf9d56913e41b5e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b28d7b346448abaf9d56913e41b5e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44763 } } }
I20260926 21:27:58.499373 32148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [sys.catalog]: This master's current role is: LEADER
I20260926 21:27:58.498893 32147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94b28d7b346448abaf9d56913e41b5e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94b28d7b346448abaf9d56913e41b5e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44763 } } }
I20260926 21:27:58.499907 32147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8 [sys.catalog]: This master's current role is: LEADER
I20260926 21:27:58.501497 32153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:27:58.506469 32153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:27:58.516355 32153 catalog_manager.cc:1347] Generated new cluster ID: 79b2940a60cd482a97147cc054d10171
I20260926 21:27:58.516546 32153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:27:58.548897 32153 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:27:58.551017 32153 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:27:58.559579 32153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8: Generated new TSK 0
I20260926 21:27:58.561177 32153 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:27:58.637279 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44763
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:27:59.005076 32167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:59.005415 32167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:59.005522 32167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:59.013266 32167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:59.013437 32167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:59.013572 32167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:27:59.025552 32167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:59.027024 32167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:59.028385 32167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:59.042860 32173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:59.042893 32175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:59.043452 32172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:59.043854 32167 server_base.cc:1034] running on GCE node
I20260926 21:27:59.044638 32167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:59.045708 32167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:59.046945 32167 hybrid_clock.cc:648] HybridClock initialized: now 1790458079046870 us; error 58 us; skew 500 ppm
I20260926 21:27:59.047331 32167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:59 dist-test-slave-4d5t krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458079, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:59.051242 32167 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:27:59.057667 32167 webserver.cc:460] Webserver started at http://127.14.142.1:38707/ using document root <none> and password file <none>
I20260926 21:27:59.059036 32167 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:59.059229 32167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:59.059646 32167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:27:59.063192 32167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "e6387f6f0f084e8589c64a841224cf8a"
format_stamp: "Formatted at 2026-09-26 21:27:59 on dist-test-slave-4d5t"
server_key: "4b088f1a8011f474fd2754feb1d8844b"
server_key_iv: "96a71272c47cd8761ba137c7775fde7c"
server_key_version: "encryptionkey@0"
I20260926 21:27:59.064049 32167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "e6387f6f0f084e8589c64a841224cf8a"
format_stamp: "Formatted at 2026-09-26 21:27:59 on dist-test-slave-4d5t"
server_key: "4b088f1a8011f474fd2754feb1d8844b"
server_key_iv: "96a71272c47cd8761ba137c7775fde7c"
server_key_version: "encryptionkey@0"
I20260926 21:27:59.070242 32167 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260926 21:27:59.074225 32182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:59.075872 32167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:27:59.076061 32167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "e6387f6f0f084e8589c64a841224cf8a"
format_stamp: "Formatted at 2026-09-26 21:27:59 on dist-test-slave-4d5t"
server_key: "4b088f1a8011f474fd2754feb1d8844b"
server_key_iv: "96a71272c47cd8761ba137c7775fde7c"
server_key_version: "encryptionkey@0"
I20260926 21:27:59.076287 32167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:59.096290 32167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:59.101735 32167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:59.102087 32167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:59.103164 32167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:59.104817 32167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:59.105047 32167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:59.105309 32167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:59.105396 32167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:59.141634 32167 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:33221
I20260926 21:27:59.141683 32294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:33221 every 8 connection(s)
I20260926 21:27:59.142968 32167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:27:59.146359 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 32167
I20260926 21:27:59.146616 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:27:59.147269 14904 external_mini_cluster.cc:1428] Setting key 6122a530aa3bde5ed70d7ed49bf2ae61
Sep 26 21:27:59 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458079, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:27:59.166409 32297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:59.146048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:40351 (local address 127.14.142.62:44763)
0926 21:27:59.146638 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:59.146645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:59.147793 (+ 1148us) server_negotiation.cc:407] Connection header received
0926 21:27:59.150114 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:59.150123 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:59.150223 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:59.150341 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:59.151922 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.153112 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.155071 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.155510 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.155666 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:59.159795 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:59.159800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:59.159804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:59.159842 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:59.162633 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:59.163704 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:59.163709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:59.163711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:59.163859 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:59.164432 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:59.164436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:59.164438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:59.164816 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:59.164966 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:59.165941 (+ 975us) server_negotiation.cc:299] Negotiation successful
0926 21:27:59.166141 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":282,"threads_started":1}
I20260926 21:27:59.167999 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:59.146613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44763 (local address 127.14.142.1:40351)
0926 21:27:59.147591 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:59.147648 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:59.149819 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:59.150576 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:59.150586 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:59.151133 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:59.151739 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:59.151757 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.153292 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.153296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:59.154895 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:59.154909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.155773 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.155783 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:59.156045 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:59.156828 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:59.156857 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:59.159473 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:59.162808 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:59.162820 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:59.162843 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:59.163512 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:59.163992 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:59.164000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:59.164006 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:59.164294 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:59.164988 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:59.165000 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:59.165500 (+ 500us) client_negotiation.cc:769] Sending connection context
0926 21:27:59.165836 (+ 336us) client_negotiation.cc:240] Negotiation successful
0926 21:27:59.166480 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":393,"threads_started":1}
I20260926 21:27:59.169934 32295 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44763
I20260926 21:27:59.170260 32295 heartbeater.cc:464] Registering TS with master...
I20260926 21:27:59.171020 32295 heartbeater.cc:511] Master 127.14.142.62:44763 requested a full tablet report, sending...
I20260926 21:27:59.173463 32106 ts_manager.cc:194] Registered new tserver with Master: e6387f6f0f084e8589c64a841224cf8a (127.14.142.1:33221)
I20260926 21:27:59.175642 32106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:40351
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:27:59.211900 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44763
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:27:59.566787 32301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:27:59.567103 32301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:27:59.567261 32301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:27:59.575068 32301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:27:59.575197 32301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:27:59.575369 32301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:27:59.585994 32301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:27:59.587689 32301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:27:59.589277 32301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:27:59.605253 32306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:59.605170 32309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:27:59.605221 32307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:27:59.606712 32301 server_base.cc:1034] running on GCE node
I20260926 21:27:59.607721 32301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:27:59.609298 32301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:27:59.610574 32301 hybrid_clock.cc:648] HybridClock initialized: now 1790458079610525 us; error 49 us; skew 500 ppm
I20260926 21:27:59.610975 32301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:27:59 dist-test-slave-4d5t krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458079, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:27:59.615044 32301 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:27:59.616935 32301 webserver.cc:460] Webserver started at http://127.14.142.2:35201/ using document root <none> and password file <none>
I20260926 21:27:59.617714 32301 fs_manager.cc:359] Metadata directory not provided
I20260926 21:27:59.617878 32301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:27:59.618225 32301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:27:59.621546 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "211a5004be614e4cb8b2c95e7f9b4170"
format_stamp: "Formatted at 2026-09-26 21:27:59 on dist-test-slave-4d5t"
server_key: "4e0d9bb58238add34684c0e34d977a96"
server_key_iv: "9066fd29be654a6a2589a113170db570"
server_key_version: "encryptionkey@0"
I20260926 21:27:59.622412 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "211a5004be614e4cb8b2c95e7f9b4170"
format_stamp: "Formatted at 2026-09-26 21:27:59 on dist-test-slave-4d5t"
server_key: "4e0d9bb58238add34684c0e34d977a96"
server_key_iv: "9066fd29be654a6a2589a113170db570"
server_key_version: "encryptionkey@0"
I20260926 21:27:59.628244 32301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 21:27:59.632265 32316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:59.633855 32301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:27:59.634066 32301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "211a5004be614e4cb8b2c95e7f9b4170"
format_stamp: "Formatted at 2026-09-26 21:27:59 on dist-test-slave-4d5t"
server_key: "4e0d9bb58238add34684c0e34d977a96"
server_key_iv: "9066fd29be654a6a2589a113170db570"
server_key_version: "encryptionkey@0"
I20260926 21:27:59.634300 32301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:27:59.655256 32301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:27:59.660045 32301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:27:59.660359 32301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:27:59.661458 32301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:27:59.662881 32301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:27:59.662987 32301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:59.663148 32301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:27:59.663221 32301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:27:59.703503 32301 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:36669
I20260926 21:27:59.703562 32428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:36669 every 8 connection(s)
I20260926 21:27:59.704965 32301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:27:59.710899 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 32301
I20260926 21:27:59.711127 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:27:59.711680 14904 external_mini_cluster.cc:1428] Setting key 6427b19fa81287f96caeeac967bd50bc
Sep 26 21:27:59 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458079, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:27:59.728593 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:59.707711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:35043 (local address 127.14.142.62:44763)
0926 21:27:59.708177 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 21:27:59.708182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:27:59.709215 (+ 1033us) server_negotiation.cc:407] Connection header received
0926 21:27:59.711312 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:27:59.711318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:27:59.711416 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:27:59.711555 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:27:59.713324 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.714157 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.716174 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.716731 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.716914 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:59.721147 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:27:59.721152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:27:59.721156 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:27:59.721195 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:27:59.724459 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:59.725489 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:59.725496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:59.725500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:59.725659 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:27:59.726170 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:27:59.726180 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:27:59.726183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:27:59.726481 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:27:59.726655 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 21:27:59.727467 (+ 812us) server_negotiation.cc:299] Negotiation successful
0926 21:27:59.727945 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":226,"threads_started":1}
I20260926 21:27:59.729758 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:27:59.708224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44763 (local address 127.14.142.2:35043)
0926 21:27:59.709037 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0926 21:27:59.709077 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 21:27:59.710961 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:27:59.711854 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:27:59.711874 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:27:59.712437 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:27:59.713142 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:59.713161 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.714344 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.714348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:59.715965 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:27:59.715982 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:27:59.717036 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:27:59.717045 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:27:59.717314 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:27:59.718145 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:27:59.718176 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:27:59.720831 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:27:59.724763 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:59.724773 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:59.724790 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:59.725310 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:59.725793 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:27:59.725799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:27:59.725803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:27:59.726027 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:27:59.726625 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:27:59.726635 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:27:59.727033 (+ 398us) client_negotiation.cc:769] Sending connection context
0926 21:27:59.727355 (+ 322us) client_negotiation.cc:240] Negotiation successful
0926 21:27:59.728397 (+ 1042us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":368,"threads_started":1}
I20260926 21:27:59.731467 32429 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44763
I20260926 21:27:59.731822 32429 heartbeater.cc:464] Registering TS with master...
I20260926 21:27:59.732688 32429 heartbeater.cc:511] Master 127.14.142.62:44763 requested a full tablet report, sending...
I20260926 21:27:59.734371 32106 ts_manager.cc:194] Registered new tserver with Master: 211a5004be614e4cb8b2c95e7f9b4170 (127.14.142.2:36669)
I20260926 21:27:59.735898 32106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:35043
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:27:59.777118 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44763
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:00.153091 32435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:00.153434 32435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:00.153563 32435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:00.161056 32435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:00.161173 32435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:00.161293 32435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:28:00.171157 32435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:44763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:00.172619 32435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:00.173957 32435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260926 21:28:00.179431 32295 heartbeater.cc:503] Master 127.14.142.62:44763 was elected leader, sending a full tablet report...
W20260926 21:28:00.190004 32441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:00.189978 32440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:00.190055 32443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:00.190819 32435 server_base.cc:1034] running on GCE node
I20260926 21:28:00.191490 32435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:00.192543 32435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:00.193850 32435 hybrid_clock.cc:648] HybridClock initialized: now 1790458080193809 us; error 43 us; skew 500 ppm
I20260926 21:28:00.194218 32435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:00 dist-test-slave-4d5t krb5kdc[32061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458080, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:00.201417 32435 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:28:00.203308 32435 webserver.cc:460] Webserver started at http://127.14.142.3:35347/ using document root <none> and password file <none>
I20260926 21:28:00.204068 32435 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:00.204246 32435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:00.204634 32435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:00.207726 32435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "cd9933f619b54f0fbf088de7b06dff95"
format_stamp: "Formatted at 2026-09-26 21:28:00 on dist-test-slave-4d5t"
server_key: "637efbaaefa9ef707ac68d46ba65dbf8"
server_key_iv: "3962455883c36f291a374ab2667b732b"
server_key_version: "encryptionkey@0"
I20260926 21:28:00.208616 32435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "cd9933f619b54f0fbf088de7b06dff95"
format_stamp: "Formatted at 2026-09-26 21:28:00 on dist-test-slave-4d5t"
server_key: "637efbaaefa9ef707ac68d46ba65dbf8"
server_key_iv: "3962455883c36f291a374ab2667b732b"
server_key_version: "encryptionkey@0"
I20260926 21:28:00.214073 32435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 21:28:00.217788 32450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:00.219206 32435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:28:00.219395 32435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "cd9933f619b54f0fbf088de7b06dff95"
format_stamp: "Formatted at 2026-09-26 21:28:00 on dist-test-slave-4d5t"
server_key: "637efbaaefa9ef707ac68d46ba65dbf8"
server_key_iv: "3962455883c36f291a374ab2667b732b"
server_key_version: "encryptionkey@0"
I20260926 21:28:00.219622 32435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:00.241436 32435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:00.247125 32435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:00.247467 32435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:00.248540 32435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:00.250013 32435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:00.250090 32435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:00.250178 32435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:00.250222 32435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:00.288508 32435 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:38141
I20260926 21:28:00.288648 32562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:38141 every 8 connection(s)
I20260926 21:28:00.289888 32435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:28:00.297308 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 32435
I20260926 21:28:00.297557 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:28:00.298138 14904 external_mini_cluster.cc:1428] Setting key 4954d180c583c55a50eca76c904ff1d2
Sep 26 21:28:00 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458080, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:00.315429 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.293086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:56651 (local address 127.14.142.62:44763)
0926 21:28:00.293727 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.293732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.294999 (+ 1267us) server_negotiation.cc:407] Connection header received
0926 21:28:00.297278 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.297291 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.297395 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.297537 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:00.299787 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.300735 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.302713 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.303171 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.303333 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.308018 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:00.308024 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:00.308028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:00.308067 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:00.310833 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.312086 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.312096 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.312098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.312273 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.313025 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.313029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.313032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.313535 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:00.313669 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.314864 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.315108 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":393,"threads_started":1}
I20260926 21:28:00.316469 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.293889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:44763 (local address 127.14.142.3:56651)
0926 21:28:00.294811 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.294857 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.296846 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.297861 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.297875 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.298670 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:00.299450 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.299471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.300930 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.300934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.302502 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.302541 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.303445 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.303453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.303750 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.304499 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:00.304530 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:00.307668 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:00.311064 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:00.311073 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:00.311093 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:00.311774 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:00.312484 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:00.312491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:00.312497 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:00.312873 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:00.313701 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:00.313712 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:00.314247 (+ 535us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.314584 (+ 337us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.315158 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":447,"threads_started":1}
I20260926 21:28:00.318043 32563 heartbeater.cc:347] Connected to a master server at 127.14.142.62:44763
I20260926 21:28:00.318666 32563 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:00.319751 32563 heartbeater.cc:511] Master 127.14.142.62:44763 requested a full tablet report, sending...
I20260926 21:28:00.321708 32106 ts_manager.cc:194] Registered new tserver with Master: cd9933f619b54f0fbf088de7b06dff95 (127.14.142.3:38141)
I20260926 21:28:00.322588 32106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:56651
I20260926 21:28:00.327244 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:28:00.348338 32565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.332279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49076 (local address 127.14.142.62:44763)
0926 21:28:00.332521 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.332533 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.332996 (+ 463us) server_negotiation.cc:407] Connection header received
0926 21:28:00.333215 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.333220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.333289 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.333379 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:00.335673 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.336441 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.338172 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.338647 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.338759 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.341733 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:00.341740 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:00.341745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:00.341798 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:00.345313 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.346132 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.346137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.346140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.346286 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.346660 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.346663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.346665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.347098 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:00.347334 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.347736 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.347942 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 21:28:00.353542 32106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:28:00.362159 32106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:28:00.387184 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.377473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39108 (local address 127.14.142.1:33221)
0926 21:28:00.378706 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.378715 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.378749 (+ 34us) server_negotiation.cc:407] Connection header received
0926 21:28:00.378848 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.378857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.379138 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.379321 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.380115 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.381557 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.384978 (+ 3421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.385939 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.386113 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.386359 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.386666 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.386833 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1060,"thread_start_us":914,"threads_started":1}
I20260926 21:28:00.387279 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.377014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:33221 (local address 127.0.0.1:39108)
0926 21:28:00.377831 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.377856 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.378027 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.379320 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.379328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.379357 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.379850 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.379870 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.381720 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.381727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.384821 (+ 3094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.384833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.386152 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.386158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.386427 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.386468 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.386609 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.386814 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":62,"thread_start_us":340,"threads_started":1}
I20260926 21:28:00.388465 32578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.376441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:36669 (local address 127.0.0.1:34488)
0926 21:28:00.377784 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.377833 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.378021 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.379815 (+ 1794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.379827 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.379863 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.380321 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.380336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.382468 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.382475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.384889 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.384906 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.387178 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.387184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.387706 (+ 522us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.387744 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.387848 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.387990 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"mutex_wait_us":30,"thread_start_us":1900,"threads_started":1}
I20260926 21:28:00.389386 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.376429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.14.142.2:36669)
0926 21:28:00.377093 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.377102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.378913 (+ 1811us) server_negotiation.cc:407] Connection header received
0926 21:28:00.379197 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.379205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.379407 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.379607 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.380714 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.382226 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.385110 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.386649 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.386802 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.387018 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.388813 (+ 1795us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.389004 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":373,"threads_started":1}
I20260926 21:28:00.389740 32580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.378216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:38141 (local address 127.0.0.1:39274)
0926 21:28:00.379019 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.379042 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.379143 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.381658 (+ 2515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.381664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.381695 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.382200 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.382211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.384518 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.384524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.386866 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.386881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.388944 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.388949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.389130 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.389160 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.389253 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.389377 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":5,"thread_start_us":498,"threads_started":1}
I20260926 21:28:00.389889 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.378647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.14.142.3:38141)
0926 21:28:00.381089 (+ 2442us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.381097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.381132 (+ 35us) server_negotiation.cc:407] Connection header received
0926 21:28:00.381237 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.381245 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.381463 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.381650 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.382415 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.384345 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.387082 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.388735 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.388879 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.389111 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.389316 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.389501 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2259,"thread_start_us":2030,"threads_started":1}
I20260926 21:28:00.391690 32230 tablet_service.cc:1467] Processing CreateTablet for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a (DEFAULT_TABLE table=test-table [id=944e90f312764772bcedaf5bf06a9c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:00.393246 32230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:00.393198 32364 tablet_service.cc:1467] Processing CreateTablet for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a (DEFAULT_TABLE table=test-table [id=944e90f312764772bcedaf5bf06a9c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:00.393436 32498 tablet_service.cc:1467] Processing CreateTablet for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a (DEFAULT_TABLE table=test-table [id=944e90f312764772bcedaf5bf06a9c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:00.394668 32364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:00.394842 32498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:00.409149 32582 tablet_bootstrap.cc:492] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Bootstrap starting.
I20260926 21:28:00.411345 32583 tablet_bootstrap.cc:492] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Bootstrap starting.
I20260926 21:28:00.413275 32582 tablet_bootstrap.cc:654] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:00.413337 32584 tablet_bootstrap.cc:492] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Bootstrap starting.
I20260926 21:28:00.414764 32582 log.cc:824] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:00.415187 32583 tablet_bootstrap.cc:654] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:00.416419 32583 log.cc:824] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:00.416728 32584 tablet_bootstrap.cc:654] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:00.418041 32584 log.cc:824] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:00.418680 32582 tablet_bootstrap.cc:492] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: No bootstrap required, opened a new log
I20260926 21:28:00.418987 32582 ts_tablet_manager.cc:1397] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 21:28:00.420142 32583 tablet_bootstrap.cc:492] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: No bootstrap required, opened a new log
I20260926 21:28:00.420439 32583 ts_tablet_manager.cc:1397] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260926 21:28:00.421221 32584 tablet_bootstrap.cc:492] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: No bootstrap required, opened a new log
I20260926 21:28:00.421499 32584 ts_tablet_manager.cc:1397] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260926 21:28:00.428752 32583 raft_consensus.cc:348] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.429242 32583 raft_consensus.cc:374] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:00.429352 32583 raft_consensus.cc:729] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 211a5004be614e4cb8b2c95e7f9b4170, State: Initialized, Role: FOLLOWER
I20260926 21:28:00.430043 32583 consensus_queue.cc:260] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.430979 32582 raft_consensus.cc:348] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.431311 32429 heartbeater.cc:503] Master 127.14.142.62:44763 was elected leader, sending a full tablet report...
I20260926 21:28:00.431466 32582 raft_consensus.cc:374] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:00.431560 32582 raft_consensus.cc:729] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6387f6f0f084e8589c64a841224cf8a, State: Initialized, Role: FOLLOWER
I20260926 21:28:00.431988 32583 ts_tablet_manager.cc:1428] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260926 21:28:00.432277 32582 consensus_queue.cc:260] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.433072 32584 raft_consensus.cc:348] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.433534 32584 raft_consensus.cc:374] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:00.433631 32584 raft_consensus.cc:729] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd9933f619b54f0fbf088de7b06dff95, State: Initialized, Role: FOLLOWER
I20260926 21:28:00.434247 32582 ts_tablet_manager.cc:1428] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260926 21:28:00.434247 32584 consensus_queue.cc:260] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.435364 32563 heartbeater.cc:503] Master 127.14.142.62:44763 was elected leader, sending a full tablet report...
I20260926 21:28:00.436040 32584 ts_tablet_manager.cc:1428] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Time spent starting tablet: real 0.014s user 0.003s sys 0.011s
I20260926 21:28:00.449625 32588 raft_consensus.cc:482] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:00.449944 32588 raft_consensus.cc:504] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.451613 32588 leader_election.cc:288] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6387f6f0f084e8589c64a841224cf8a (127.14.142.1:33221), cd9933f619b54f0fbf088de7b06dff95 (127.14.142.3:38141)
W20260926 21:28:00.457274 32430 tablet.cc:2369] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:00.460454 32589 raft_consensus.cc:482] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:00.460841 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.452923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:51493 (local address 127.14.142.3:38141)
0926 21:28:00.453136 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.453142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.453493 (+ 351us) server_negotiation.cc:407] Connection header received
0926 21:28:00.453697 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.453703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.453770 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.453866 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.454517 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.455650 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.458196 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.459473 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.459643 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.459720 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.460256 (+ 536us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.460413 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:28:00.460798 32589 raft_consensus.cc:504] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.461083 32591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.451922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:33221 (local address 127.14.142.2:41875)
0926 21:28:00.452714 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.452746 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.452913 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.453530 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.453536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.453564 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.453999 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.454013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.455189 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.455196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.458108 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.458125 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.459872 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.459877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.460013 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.460068 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.460482 (+ 414us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.460663 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":77,"thread_start_us":375,"threads_started":1}
I20260926 21:28:00.461575 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.452069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:41875 (local address 127.14.142.1:33221)
0926 21:28:00.452344 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.452350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.452837 (+ 487us) server_negotiation.cc:407] Connection header received
0926 21:28:00.453123 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.453130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.453214 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.453280 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.454162 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.455063 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.458267 (+ 3204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.459494 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.459643 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.459720 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.460248 (+ 528us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.460387 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:28:00.462060 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.452818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:38141 (local address 127.14.142.2:51493)
0926 21:28:00.453378 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.453396 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.453555 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.453879 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.453883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.453911 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.454349 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.454360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.455806 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.455812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.458053 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.458069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.459704 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.459711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.459945 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.460070 (+ 125us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.461345 (+ 1275us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.461497 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":148,"thread_start_us":303,"threads_started":1}
I20260926 21:28:00.462538 32250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d2c3ce6a6a4325b45629e0a2a67c1a" candidate_uuid: "211a5004be614e4cb8b2c95e7f9b4170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6387f6f0f084e8589c64a841224cf8a" is_pre_election: true
I20260926 21:28:00.462980 32589 leader_election.cc:288] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 211a5004be614e4cb8b2c95e7f9b4170 (127.14.142.2:36669), cd9933f619b54f0fbf088de7b06dff95 (127.14.142.3:38141)
I20260926 21:28:00.463379 32518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d2c3ce6a6a4325b45629e0a2a67c1a" candidate_uuid: "211a5004be614e4cb8b2c95e7f9b4170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd9933f619b54f0fbf088de7b06dff95" is_pre_election: true
I20260926 21:28:00.463948 32518 raft_consensus.cc:2454] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 211a5004be614e4cb8b2c95e7f9b4170 in term 0.
I20260926 21:28:00.462982 32250 raft_consensus.cc:2454] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 211a5004be614e4cb8b2c95e7f9b4170 in term 0.
I20260926 21:28:00.464809 32319 leader_election.cc:302] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 211a5004be614e4cb8b2c95e7f9b4170, cd9933f619b54f0fbf088de7b06dff95; no voters:
I20260926 21:28:00.465701 32588 raft_consensus.cc:2781] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:28:00.465893 32588 raft_consensus.cc:482] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:28:00.466019 32588 raft_consensus.cc:3037] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:00.469609 32588 raft_consensus.cc:504] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.470847 32250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d2c3ce6a6a4325b45629e0a2a67c1a" candidate_uuid: "211a5004be614e4cb8b2c95e7f9b4170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6387f6f0f084e8589c64a841224cf8a"
I20260926 21:28:00.471071 32250 raft_consensus.cc:3037] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:00.471457 32588 leader_election.cc:288] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [CANDIDATE]: Term 1 election: Requested vote from peers e6387f6f0f084e8589c64a841224cf8a (127.14.142.1:33221), cd9933f619b54f0fbf088de7b06dff95 (127.14.142.3:38141)
I20260926 21:28:00.472465 32518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d2c3ce6a6a4325b45629e0a2a67c1a" candidate_uuid: "211a5004be614e4cb8b2c95e7f9b4170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd9933f619b54f0fbf088de7b06dff95"
I20260926 21:28:00.472756 32518 raft_consensus.cc:3037] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:00.474556 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.463999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:46265 (local address 127.14.142.3:38141)
0926 21:28:00.464189 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.464196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.467111 (+ 2915us) server_negotiation.cc:407] Connection header received
0926 21:28:00.467264 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.467270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.467345 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.467432 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.468171 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.469312 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.471925 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.473246 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.473401 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.473483 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.474069 (+ 586us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.474219 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 21:28:00.474603 32250 raft_consensus.cc:2454] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 211a5004be614e4cb8b2c95e7f9b4170 in term 1.
I20260926 21:28:00.474927 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.463409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:53023 (local address 127.14.142.2:36669)
0926 21:28:00.463631 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.463638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.465233 (+ 1595us) server_negotiation.cc:407] Connection header received
0926 21:28:00.465487 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.465493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.465580 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.465641 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:00.467305 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.468620 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.472039 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.473500 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.473661 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.473736 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.474465 (+ 729us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.474635 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:28:00.475052 32319 leader_election.cc:302] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 211a5004be614e4cb8b2c95e7f9b4170, e6387f6f0f084e8589c64a841224cf8a; no voters:
I20260926 21:28:00.475346 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.463955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:38141 (local address 127.14.142.1:46265)
0926 21:28:00.466989 (+ 3034us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.467009 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.467138 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.467581 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.467585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.467613 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.467968 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.467978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.469515 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.469520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.471714 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.471737 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.473685 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.473691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.473864 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.473898 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.474811 (+ 913us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.474957 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2911,"mutex_wait_us":2,"thread_start_us":777,"threads_started":1}
I20260926 21:28:00.475998 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.463428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:36669 (local address 127.14.142.1:53023)
0926 21:28:00.465129 (+ 1701us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:00.465150 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:00.465342 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:00.465933 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:00.465939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:00.466013 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:00.466429 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.466442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.468818 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.468824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.471832 (+ 3008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:00.471849 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.474042 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.474049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:00.474251 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.474290 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 21:28:00.475403 (+ 1113us) client_negotiation.cc:240] Negotiation successful
0926 21:28:00.475584 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1497,"mutex_wait_us":49,"thread_start_us":3248,"threads_started":1}
I20260926 21:28:00.476002 32517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d2c3ce6a6a4325b45629e0a2a67c1a" candidate_uuid: "e6387f6f0f084e8589c64a841224cf8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd9933f619b54f0fbf088de7b06dff95" is_pre_election: true
I20260926 21:28:00.476650 32518 raft_consensus.cc:2454] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 211a5004be614e4cb8b2c95e7f9b4170 in term 1.
I20260926 21:28:00.477427 32588 raft_consensus.cc:2781] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:00.477977 32588 raft_consensus.cc:686] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 1 LEADER]: Becoming Leader. State: Replica: 211a5004be614e4cb8b2c95e7f9b4170, State: Running, Role: LEADER
I20260926 21:28:00.477864 32384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d2c3ce6a6a4325b45629e0a2a67c1a" candidate_uuid: "e6387f6f0f084e8589c64a841224cf8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "211a5004be614e4cb8b2c95e7f9b4170" is_pre_election: true
I20260926 21:28:00.478634 32588 consensus_queue.cc:237] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } }
I20260926 21:28:00.479024 32186 leader_election.cc:302] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e6387f6f0f084e8589c64a841224cf8a; no voters: 211a5004be614e4cb8b2c95e7f9b4170, cd9933f619b54f0fbf088de7b06dff95
I20260926 21:28:00.479471 32589 raft_consensus.cc:2726] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 21:28:00.484248 32104 catalog_manager.cc:5495] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 211a5004be614e4cb8b2c95e7f9b4170 (127.14.142.2). New cstate: current_term: 1 leader_uuid: "211a5004be614e4cb8b2c95e7f9b4170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "211a5004be614e4cb8b2c95e7f9b4170" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 36669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 } health_report { overall_health: UNKNOWN } } }
I20260926 21:28:00.516275 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.509996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34498 (local address 127.14.142.2:36669)
0926 21:28:00.510387 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.510395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.510540 (+ 145us) server_negotiation.cc:407] Connection header received
0926 21:28:00.511005 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.511010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.511123 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.511295 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:28:00.511714 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.512484 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.514009 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.514426 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.514562 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.515125 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:28:00.515638 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:28:00.515812 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.515863 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.515948 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260926 21:28:00.525096 32250 raft_consensus.cc:1261] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 1 FOLLOWER]: Refusing update from remote peer 211a5004be614e4cb8b2c95e7f9b4170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:00.525146 32518 raft_consensus.cc:1261] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 1 FOLLOWER]: Refusing update from remote peer 211a5004be614e4cb8b2c95e7f9b4170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:00.526100 32595 consensus_queue.cc:1035] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6387f6f0f084e8589c64a841224cf8a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 33221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:00.526705 32588 consensus_queue.cc:1035] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd9933f619b54f0fbf088de7b06dff95" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:00.540534 32598 mvcc.cc:205] Tried to move back new op lower bound from 7333716297819623424 to 7333716297645604864. Current Snapshot: MvccSnapshot[applied={T|T < 7333716297819623424}]
I20260926 21:28:00.558223 32106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:00.558656 32106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:00.563030 32106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94b28d7b346448abaf9d56913e41b5e8: Sending DeleteTablet for 3 replicas of tablet 89d2c3ce6a6a4325b45629e0a2a67c1a
I20260926 21:28:00.564508 32364 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:00 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:34488
I20260926 21:28:00.564872 32230 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:00 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:39108
I20260926 21:28:00.566128 32498 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d2c3ce6a6a4325b45629e0a2a67c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:00 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:39274
I20260926 21:28:00.566437 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 32167
I20260926 21:28:00.581205 32610 tablet_replica.cc:330] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: stopping tablet replica
I20260926 21:28:00.581898 32610 raft_consensus.cc:2229] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:00.582600 32610 raft_consensus.cc:2258] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:00.583189 32609 tablet_replica.cc:330] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: stopping tablet replica
I20260926 21:28:00.583739 32609 raft_consensus.cc:2229] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:00.584394 32609 raft_consensus.cc:2258] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:00.585472 32608 tablet_replica.cc:330] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: stopping tablet replica
I20260926 21:28:00.585940 32608 raft_consensus.cc:2229] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:28:00.586750 32609 ts_tablet_manager.cc:1905] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:00.586735 32608 raft_consensus.cc:2258] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:00.589705 32610 ts_tablet_manager.cc:1905] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:00.591270 32608 ts_tablet_manager.cc:1905] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:00.594831 32609 ts_tablet_manager.cc:1918] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:00.595026 32609 log.cc:1196] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/89d2c3ce6a6a4325b45629e0a2a67c1a
I20260926 21:28:00.595669 32609 ts_tablet_manager.cc:1939] T 89d2c3ce6a6a4325b45629e0a2a67c1a P e6387f6f0f084e8589c64a841224cf8a: Deleting consensus metadata
Sep 26 21:28:00 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:28:00.597940 32092 catalog_manager.cc:4841] TS e6387f6f0f084e8589c64a841224cf8a (127.14.142.1:33221): tablet 89d2c3ce6a6a4325b45629e0a2a67c1a (table test-table [id=944e90f312764772bcedaf5bf06a9c06]) successfully deleted
I20260926 21:28:00.601325 32608 ts_tablet_manager.cc:1918] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:00.601537 32608 log.cc:1196] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/89d2c3ce6a6a4325b45629e0a2a67c1a
I20260926 21:28:00.602144 32608 ts_tablet_manager.cc:1939] T 89d2c3ce6a6a4325b45629e0a2a67c1a P 211a5004be614e4cb8b2c95e7f9b4170: Deleting consensus metadata
I20260926 21:28:00.604219 32093 catalog_manager.cc:4841] TS 211a5004be614e4cb8b2c95e7f9b4170 (127.14.142.2:36669): tablet 89d2c3ce6a6a4325b45629e0a2a67c1a (table test-table [id=944e90f312764772bcedaf5bf06a9c06]) successfully deleted
I20260926 21:28:00.604401 32610 ts_tablet_manager.cc:1918] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:00.604621 32610 log.cc:1196] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/89d2c3ce6a6a4325b45629e0a2a67c1a
I20260926 21:28:00.605288 32610 ts_tablet_manager.cc:1939] T 89d2c3ce6a6a4325b45629e0a2a67c1a P cd9933f619b54f0fbf088de7b06dff95: Deleting consensus metadata
I20260926 21:28:00.607353 32092 catalog_manager.cc:4841] TS cd9933f619b54f0fbf088de7b06dff95 (127.14.142.3:38141): tablet 89d2c3ce6a6a4325b45629e0a2a67c1a (table test-table [id=944e90f312764772bcedaf5bf06a9c06]) successfully deleted
I20260926 21:28:00.607517 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.568624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39120 (local address 127.14.142.1:33221)
0926 21:28:00.568839 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.568848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.578851 (+ 10003us) server_negotiation.cc:407] Connection header received
0926 21:28:00.580129 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.580138 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.580269 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.580384 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:00.584818 (+ 4434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.586172 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.588703 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.589480 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.589598 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.599299 (+ 9701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:00.599316 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:00.599336 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:00.599388 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:00.603641 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.604792 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.604807 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.604815 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.605014 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.605545 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.605554 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.605561 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.606096 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:00.606438 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.606808 (+ 370us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.607077 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:28:00.608201 32290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:00.721527 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 32167
I20260926 21:28:00.747037 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 32301
Sep 26 21:28:00 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:28:00.764539 32576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.748020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.14.142.2:36669)
0926 21:28:00.748254 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.748261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.748281 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:28:00.748348 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.748351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.748415 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.748537 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:00.749890 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.750613 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.751894 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.752356 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.752519 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.757968 (+ 5449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:00.757983 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:00.757999 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:00.758059 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:00.761467 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.762249 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.762263 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.762265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.762427 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.762842 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.762847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.762852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.763296 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:00.763629 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.763914 (+ 285us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.764188 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 21:28:00.765491 32424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:00.884095 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 32301
I20260926 21:28:00.904707 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 32435
Sep 26 21:28:00 dist-test-slave-4d5t krb5kdc[32061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:28:00.920346 32581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:00.905454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.14.142.3:38141)
0926 21:28:00.905668 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:00.905675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:00.905759 (+ 84us) server_negotiation.cc:407] Connection header received
0926 21:28:00.905978 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:00.905982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:00.906048 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:00.906157 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:00.907435 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.908137 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.909542 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:00.909943 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:00.910082 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:00.914643 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:00.914660 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:00.914676 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:00.914720 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:00.917579 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.918284 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.918292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.918294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.918418 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:00.918804 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:00.918810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:00.918813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:00.919187 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:00.919481 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:00.919797 (+ 316us) server_negotiation.cc:299] Negotiation successful
0926 21:28:00.920011 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 21:28:00.921208 32558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:01.013032 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 32435
I20260926 21:28:01.031955 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 32074
I20260926 21:28:01.033200 32136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:01.151182 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 32074
2026-09-26T21:28:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4649 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T21:28:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:01Z Disabled control of system clock
I20260926 21:28:01.245128 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41383
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 21:28:01.605865 32619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:01.606166 32619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:01.606249 32619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:01.613492 32619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:01.613672 32619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:01.613740 32619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:01.613806 32619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:01.613843 32619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:01.623018 32619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41383
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:01.624416 32619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:01.626195 32619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:01.639328 32625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:01.639314 32627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:01.639561 32624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:01.640298 32619 server_base.cc:1034] running on GCE node
I20260926 21:28:01.641263 32619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:01.642869 32619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:01.644155 32619 hybrid_clock.cc:648] HybridClock initialized: now 1790458081644103 us; error 66 us; skew 500 ppm
I20260926 21:28:01.644536 32619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:01.647001 32619 webserver.cc:460] Webserver started at http://127.14.142.62:39805/ using document root <none> and password file <none>
I20260926 21:28:01.647696 32619 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:01.647825 32619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:01.648150 32619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:01.651129 32619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "f16ad9a638134a0ea3186027eb290792"
format_stamp: "Formatted at 2026-09-26 21:28:01 on dist-test-slave-4d5t"
server_key: "2b7c4160298c4883498b075e72058f99"
server_key_iv: "cf53044938760ad0ec42e93fcec6f4d1"
server_key_version: "encryptionkey@0"
I20260926 21:28:01.651952 32619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "f16ad9a638134a0ea3186027eb290792"
format_stamp: "Formatted at 2026-09-26 21:28:01 on dist-test-slave-4d5t"
server_key: "2b7c4160298c4883498b075e72058f99"
server_key_iv: "cf53044938760ad0ec42e93fcec6f4d1"
server_key_version: "encryptionkey@0"
I20260926 21:28:01.657328 32619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260926 21:28:01.660876 32633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:01.662220 32619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:28:01.662427 32619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "f16ad9a638134a0ea3186027eb290792"
format_stamp: "Formatted at 2026-09-26 21:28:01 on dist-test-slave-4d5t"
server_key: "2b7c4160298c4883498b075e72058f99"
server_key_iv: "cf53044938760ad0ec42e93fcec6f4d1"
server_key_version: "encryptionkey@0"
I20260926 21:28:01.662659 32619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:01.692875 32619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:01.693778 32619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:01.694092 32619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:01.714141 32619 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:41383
I20260926 21:28:01.714160 32684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:41383 every 8 connection(s)
I20260926 21:28:01.715531 32619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:01.715885 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 32619
I20260926 21:28:01.716064 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:01.716672 14904 external_mini_cluster.cc:1428] Setting key 01566b4a03a662a963a12d74582fa5b3
I20260926 21:28:01.720679 32685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:01.731350 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:01.719192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.14.142.62:41383)
0926 21:28:01.720183 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:01.720194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:01.720237 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:28:01.722043 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:01.722064 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:01.722491 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:01.722867 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:01.723452 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:01.724985 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:01.726783 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:01.727456 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:01.727650 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:01.729023 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:01.729031 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:01.729048 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:01.729667 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:01.729749 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:01.729769 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:01.729906 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:01.730139 (+ 233us) server_negotiation.cc:299] Negotiation successful
0926 21:28:01.730430 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":367,"threads_started":1}
I20260926 21:28:01.731673 32685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792: Bootstrap starting.
I20260926 21:28:01.735353 32685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:01.736375 32685 log.cc:824] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:01.739122 32685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792: No bootstrap required, opened a new log
I20260926 21:28:01.746906 32685 raft_consensus.cc:348] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16ad9a638134a0ea3186027eb290792" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41383 } }
I20260926 21:28:01.747336 32685 raft_consensus.cc:374] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:01.747447 32685 raft_consensus.cc:729] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16ad9a638134a0ea3186027eb290792, State: Initialized, Role: FOLLOWER
I20260926 21:28:01.748158 32685 consensus_queue.cc:260] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16ad9a638134a0ea3186027eb290792" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41383 } }
I20260926 21:28:01.748422 32685 raft_consensus.cc:388] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:01.748607 32685 raft_consensus.cc:482] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:01.748836 32685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:01.751068 32685 raft_consensus.cc:504] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16ad9a638134a0ea3186027eb290792" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41383 } }
I20260926 21:28:01.751535 32685 leader_election.cc:302] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f16ad9a638134a0ea3186027eb290792; no voters:
I20260926 21:28:01.752274 32685 leader_election.cc:288] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:01.752454 32690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:01.753634 32690 raft_consensus.cc:686] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [term 1 LEADER]: Becoming Leader. State: Replica: f16ad9a638134a0ea3186027eb290792, State: Running, Role: LEADER
I20260926 21:28:01.754328 32690 consensus_queue.cc:237] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16ad9a638134a0ea3186027eb290792" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41383 } }
I20260926 21:28:01.755259 32685 sys_catalog.cc:564] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:01.757447 32692 sys_catalog.cc:455] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f16ad9a638134a0ea3186027eb290792. Latest consensus state: current_term: 1 leader_uuid: "f16ad9a638134a0ea3186027eb290792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16ad9a638134a0ea3186027eb290792" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41383 } } }
I20260926 21:28:01.757961 32692 sys_catalog.cc:458] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:01.757815 32691 sys_catalog.cc:455] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f16ad9a638134a0ea3186027eb290792" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16ad9a638134a0ea3186027eb290792" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41383 } } }
I20260926 21:28:01.758169 32691 sys_catalog.cc:458] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:01.761046 32695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:01.767089 32695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:01.777415 32695 catalog_manager.cc:1347] Generated new cluster ID: 3e5d3e4aeff04c64b899048007d92b89
I20260926 21:28:01.777611 32695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:01.785658 32695 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:01.787245 32695 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:01.797343 32695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792: Generated new TSK 0
I20260926 21:28:01.798254 32695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:01.815795 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41383
--builtin_ntp_servers=127.14.142.20:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 21:28:02.198989 32709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:02.199299 32709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:02.199461 32709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:02.207113 32709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:02.207222 32709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:02.207405 32709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:28:02.217294 32709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:02.218768 32709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:02.221071 32709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:02.237926 32714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:02.238000 32717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:02.238844 32715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:02.239238 32709 server_base.cc:1034] running on GCE node
I20260926 21:28:02.239835 32709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:02.240844 32709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:02.242084 32709 hybrid_clock.cc:648] HybridClock initialized: now 1790458082242032 us; error 41 us; skew 500 ppm
I20260926 21:28:02.242435 32709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:02.245101 32709 webserver.cc:460] Webserver started at http://127.14.142.1:38257/ using document root <none> and password file <none>
I20260926 21:28:02.245874 32709 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:02.246021 32709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:02.246367 32709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:02.249784 32709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "fd6de8b563f342b1b8986255abba8f64"
format_stamp: "Formatted at 2026-09-26 21:28:02 on dist-test-slave-4d5t"
server_key: "2cabb55c8fb8e2cedce5c15319a45364"
server_key_iv: "746018c3d5a9dfcb85276babc3570c39"
server_key_version: "encryptionkey@0"
I20260926 21:28:02.250710 32709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "fd6de8b563f342b1b8986255abba8f64"
format_stamp: "Formatted at 2026-09-26 21:28:02 on dist-test-slave-4d5t"
server_key: "2cabb55c8fb8e2cedce5c15319a45364"
server_key_iv: "746018c3d5a9dfcb85276babc3570c39"
server_key_version: "encryptionkey@0"
I20260926 21:28:02.258466 32709 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260926 21:28:02.262475 32723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:02.264498 32709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 21:28:02.264778 32709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "fd6de8b563f342b1b8986255abba8f64"
format_stamp: "Formatted at 2026-09-26 21:28:02 on dist-test-slave-4d5t"
server_key: "2cabb55c8fb8e2cedce5c15319a45364"
server_key_iv: "746018c3d5a9dfcb85276babc3570c39"
server_key_version: "encryptionkey@0"
I20260926 21:28:02.265080 32709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:02.295564 32709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:02.296702 32709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:02.297040 32709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:02.298614 32709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:02.300711 32709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:02.300838 32709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:02.300974 32709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:02.301080 32709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:02.337922 32709 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:37837
I20260926 21:28:02.338045 368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:37837 every 8 connection(s)
I20260926 21:28:02.339267 32709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:28:02.349666 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 32709
I20260926 21:28:02.349906 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:28:02.350452 14904 external_mini_cluster.cc:1428] Setting key 06819f76a592c8e4f6cfeb79338e794e
I20260926 21:28:02.353518 371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:02.342111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:47663 (local address 127.14.142.62:41383)
0926 21:28:02.342579 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:02.342586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:02.343810 (+ 1224us) server_negotiation.cc:407] Connection header received
0926 21:28:02.346547 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:02.346562 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:02.346649 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:02.346777 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:02.348040 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.348852 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.350636 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.351025 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.351168 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:02.352650 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:02.352655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:02.352658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:02.352675 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:02.352713 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:02.352724 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:02.352821 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:02.353061 (+ 240us) server_negotiation.cc:299] Negotiation successful
0926 21:28:02.353158 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":219,"threads_started":1}
I20260926 21:28:02.353858 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41383
--builtin_ntp_servers=127.14.142.20:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 21:28:02.354630 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:02.342696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41383 (local address 127.14.142.1:47663)
0926 21:28:02.343648 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:02.343694 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:02.346096 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:02.347118 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:02.347128 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:02.347236 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:02.347852 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:02.347870 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.349009 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.349013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:02.350442 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:02.350454 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.351268 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.351276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:02.351505 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:02.352283 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:02.352288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:02.352344 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:02.352434 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:02.352833 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:02.352846 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:02.352861 (+ 15us) client_negotiation.cc:769] Sending connection context
0926 21:28:02.353052 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 21:28:02.353483 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":285,"threads_started":1}
I20260926 21:28:02.356315 369 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41383
I20260926 21:28:02.356776 369 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:02.357664 369 heartbeater.cc:511] Master 127.14.142.62:41383 requested a full tablet report, sending...
I20260926 21:28:02.360502 32650 ts_manager.cc:194] Registered new tserver with Master: fd6de8b563f342b1b8986255abba8f64 (127.14.142.1:37837)
I20260926 21:28:02.362802 32650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.1:47663
W20260926 21:28:02.720196 373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:02.720671 373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:02.720847 373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:02.734074 373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:02.734216 373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:02.734413 373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:28:02.745694 373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:02.746964 373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:02.748733 373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:02.762090 381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:02.762187 379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:02.762580 378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:02.763187 373 server_base.cc:1034] running on GCE node
I20260926 21:28:02.763798 373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:02.764871 373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:02.766089 373 hybrid_clock.cc:648] HybridClock initialized: now 1790458082766016 us; error 59 us; skew 500 ppm
I20260926 21:28:02.766417 373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:02.769021 373 webserver.cc:460] Webserver started at http://127.14.142.2:43331/ using document root <none> and password file <none>
I20260926 21:28:02.769761 373 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:02.769937 373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:02.770223 373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:02.773406 373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "cd0eb1591bd744a780630a77663ff606"
format_stamp: "Formatted at 2026-09-26 21:28:02 on dist-test-slave-4d5t"
server_key: "e2ce1679d918952b8c393606bf9044fe"
server_key_iv: "1d6324022d7ece9276dbced792f78a6f"
server_key_version: "encryptionkey@0"
I20260926 21:28:02.774192 373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "cd0eb1591bd744a780630a77663ff606"
format_stamp: "Formatted at 2026-09-26 21:28:02 on dist-test-slave-4d5t"
server_key: "e2ce1679d918952b8c393606bf9044fe"
server_key_iv: "1d6324022d7ece9276dbced792f78a6f"
server_key_version: "encryptionkey@0"
I20260926 21:28:02.779699 373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 21:28:02.783417 387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:02.785048 373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:28:02.785234 373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "cd0eb1591bd744a780630a77663ff606"
format_stamp: "Formatted at 2026-09-26 21:28:02 on dist-test-slave-4d5t"
server_key: "e2ce1679d918952b8c393606bf9044fe"
server_key_iv: "1d6324022d7ece9276dbced792f78a6f"
server_key_version: "encryptionkey@0"
I20260926 21:28:02.785473 373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:02.812073 373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:02.813128 373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:02.813604 373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:02.814723 373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:02.816370 373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:02.816469 373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:02.816612 373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:02.816701 373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260926 21:28:02.854338 373 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:39143
I20260926 21:28:02.854442 500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:39143 every 8 connection(s)
I20260926 21:28:02.855705 373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:28:02.864492 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 373
I20260926 21:28:02.864800 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:28:02.865319 14904 external_mini_cluster.cc:1428] Setting key c8e43c53f332bf01a6131c2c95ba6ed4
I20260926 21:28:02.869498 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41383
--builtin_ntp_servers=127.14.142.20:40699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 21:28:02.871811 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:02.858690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:52979 (local address 127.14.142.62:41383)
0926 21:28:02.859168 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:02.859172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:02.860305 (+ 1133us) server_negotiation.cc:407] Connection header received
0926 21:28:02.862246 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:02.862253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:02.862358 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:02.862477 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:02.864167 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.865274 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.868126 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.868811 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.868961 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:02.870589 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:02.870599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:02.870606 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:02.870635 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:02.870689 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:02.870709 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:02.870810 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:02.871147 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 21:28:02.871344 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":269,"threads_started":1}
I20260926 21:28:02.872785 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:02.859186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41383 (local address 127.14.142.2:52979)
0926 21:28:02.860117 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:02.860152 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:02.861942 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:02.862721 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:02.862730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:02.862828 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:02.863774 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:02.863805 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.865488 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.865496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:02.867916 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:02.867939 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:02.869084 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:02.869090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:02.869327 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:02.870210 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:02.870213 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:02.870258 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:02.870361 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:02.870821 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:02.870830 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:02.870880 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 21:28:02.871074 (+ 194us) client_negotiation.cc:240] Negotiation successful
0926 21:28:02.871492 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":299,"threads_started":1}
I20260926 21:28:02.874351 501 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41383
I20260926 21:28:02.874722 501 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:02.875972 501 heartbeater.cc:511] Master 127.14.142.62:41383 requested a full tablet report, sending...
I20260926 21:28:02.878497 32650 ts_manager.cc:194] Registered new tserver with Master: cd0eb1591bd744a780630a77663ff606 (127.14.142.2:39143)
I20260926 21:28:02.879410 32650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.2:52979
W20260926 21:28:03.227880 505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:03.228204 505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:03.228292 505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:03.236040 505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:03.236147 505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:03.236263 505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:28:03.245920 505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:41383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:03.247285 505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:03.249079 505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:03.266530 513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:03.267093 511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:03.266502 510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:03.267865 505 server_base.cc:1034] running on GCE node
I20260926 21:28:03.268800 505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:03.270340 505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:03.271626 505 hybrid_clock.cc:648] HybridClock initialized: now 1790458083271575 us; error 62 us; skew 500 ppm
I20260926 21:28:03.271988 505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:03.274597 505 webserver.cc:460] Webserver started at http://127.14.142.3:44227/ using document root <none> and password file <none>
I20260926 21:28:03.275329 505 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:03.275427 505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:03.275702 505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:03.279023 505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "c43e58d835594152825a6dc8c7a0d372"
format_stamp: "Formatted at 2026-09-26 21:28:03 on dist-test-slave-4d5t"
server_key: "05fd5c183307bf250c42e1663c27189b"
server_key_iv: "56727d559a91ed5f9bcd2923da5ea387"
server_key_version: "encryptionkey@0"
I20260926 21:28:03.280005 505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "c43e58d835594152825a6dc8c7a0d372"
format_stamp: "Formatted at 2026-09-26 21:28:03 on dist-test-slave-4d5t"
server_key: "05fd5c183307bf250c42e1663c27189b"
server_key_iv: "56727d559a91ed5f9bcd2923da5ea387"
server_key_version: "encryptionkey@0"
I20260926 21:28:03.286536 505 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 21:28:03.291169 519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:03.293119 505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:28:03.293346 505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "c43e58d835594152825a6dc8c7a0d372"
format_stamp: "Formatted at 2026-09-26 21:28:03 on dist-test-slave-4d5t"
server_key: "05fd5c183307bf250c42e1663c27189b"
server_key_iv: "56727d559a91ed5f9bcd2923da5ea387"
server_key_version: "encryptionkey@0"
I20260926 21:28:03.293593 505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:03.321734 505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:03.322801 505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:03.323155 505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:03.324265 505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:03.325847 505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:03.325982 505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:03.326130 505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:03.326233 505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:03.365103 505 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:33213
I20260926 21:28:03.365166 632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:33213 every 8 connection(s)
I20260926 21:28:03.366362 369 heartbeater.cc:503] Master 127.14.142.62:41383 was elected leader, sending a full tablet report...
I20260926 21:28:03.366562 505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:28:03.372300 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 505
I20260926 21:28:03.372488 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:28:03.373032 14904 external_mini_cluster.cc:1428] Setting key 2fd77632192d950f2668cb4c160d32b1
I20260926 21:28:03.381564 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.369697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:41117 (local address 127.14.142.62:41383)
0926 21:28:03.369900 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.369907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.371403 (+ 1496us) server_negotiation.cc:407] Connection header received
0926 21:28:03.373658 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.373665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.373765 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.373852 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:03.375189 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.376209 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.377991 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.378384 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.378489 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.380383 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:03.380388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:03.380391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:03.380408 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:03.380442 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:03.380454 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:03.380539 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.380994 (+ 455us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.381127 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 21:28:03.382360 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.370227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:41383 (local address 127.14.142.3:41117)
0926 21:28:03.371233 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:03.371269 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:03.373240 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:03.374106 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:03.374115 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:03.374263 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:03.375006 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.375026 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.376377 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.376382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.377825 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.377838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.378611 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.378616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.378835 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.379746 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:03.379750 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:03.379818 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:03.380016 (+ 198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:03.380651 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:03.380660 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:03.380667 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 21:28:03.380852 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 21:28:03.381187 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":323,"threads_started":1}
I20260926 21:28:03.383831 633 heartbeater.cc:347] Connected to a master server at 127.14.142.62:41383
I20260926 21:28:03.384220 633 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:03.385067 633 heartbeater.cc:511] Master 127.14.142.62:41383 requested a full tablet report, sending...
I20260926 21:28:03.386726 32650 ts_manager.cc:194] Registered new tserver with Master: c43e58d835594152825a6dc8c7a0d372 (127.14.142.3:33213)
I20260926 21:28:03.388042 32650 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.3:41117
I20260926 21:28:03.389268 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:28:03.401089 503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.394231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.14.142.62:41383)
0926 21:28:03.394413 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.394417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.394973 (+ 556us) server_negotiation.cc:407] Connection header received
0926 21:28:03.395179 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.395183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.395245 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.395334 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:03.395814 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.396781 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.398520 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.399048 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.399154 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.400354 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:03.400358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:03.400361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:03.400379 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:03.400418 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:03.400436 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:03.400507 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.400727 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.400827 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 21:28:03.405174 32649 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:28:03.413681 32649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:28:03.443899 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.427952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.14.142.1:37837)
0926 21:28:03.428617 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.428627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.429635 (+ 1008us) server_negotiation.cc:407] Connection header received
0926 21:28:03.430394 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.430405 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.430687 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.430871 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:03.432230 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.434096 (+ 1866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.438929 (+ 4833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.441668 (+ 2739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.441862 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.442278 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.443319 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.443512 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":306,"threads_started":1}
I20260926 21:28:03.444809 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.427901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50910)
0926 21:28:03.429417 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:03.429474 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:03.429834 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:03.431243 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:03.431255 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:03.431303 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:03.431977 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.431992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.434288 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.434292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.438591 (+ 4299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.438614 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.442580 (+ 3966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.442590 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.442955 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.443028 (+ 73us) client_negotiation.cc:769] Sending connection context
0926 21:28:03.444220 (+ 1192us) client_negotiation.cc:240] Negotiation successful
0926 21:28:03.444457 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"thread_start_us":1029,"threads_started":1}
I20260926 21:28:03.451125 304 tablet_service.cc:1467] Processing CreateTablet for tablet ab6784394e164d548ff6c6d3572160bc (DEFAULT_TABLE table=test-table [id=d934f64a79fa41b9af307c4dd270527a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:03.452756 304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6784394e164d548ff6c6d3572160bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:03.454674 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.428773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:39143 (local address 127.0.0.1:54814)
0926 21:28:03.445144 (+ 16371us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:03.445171 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:03.445305 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:03.446377 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:03.446381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:03.446407 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:03.446614 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.446629 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.448460 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.448465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.450379 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.450395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.453865 (+ 3470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.453872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.454102 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.454139 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 21:28:03.454208 (+ 69us) client_negotiation.cc:240] Negotiation successful
0926 21:28:03.454362 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16198,"threads_started":1}
I20260926 21:28:03.455803 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.428957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54814 (local address 127.14.142.2:39143)
0926 21:28:03.445844 (+ 16887us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.445853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.445889 (+ 36us) server_negotiation.cc:407] Connection header received
0926 21:28:03.445973 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.445979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.446184 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.446394 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:03.446799 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.448270 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.452200 (+ 3930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.453669 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.454885 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.455161 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.455234 (+ 73us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.455456 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16737,"thread_start_us":16617,"threads_started":1}
I20260926 21:28:03.464265 436 tablet_service.cc:1467] Processing CreateTablet for tablet ab6784394e164d548ff6c6d3572160bc (DEFAULT_TABLE table=test-table [id=d934f64a79fa41b9af307c4dd270527a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:03.465895 436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6784394e164d548ff6c6d3572160bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:03.469584 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.431442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:33213 (local address 127.0.0.1:38728)
0926 21:28:03.447045 (+ 15603us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:03.447063 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:03.447168 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:03.460744 (+ 13576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:03.460752 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:03.460785 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:03.461200 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.461213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.463189 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.463195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.465293 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.465311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.468752 (+ 3441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.468762 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.468981 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.469019 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:28:03.469107 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 21:28:03.469253 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15456,"thread_start_us":15308,"threads_started":1}
I20260926 21:28:03.470217 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.430796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.14.142.3:33213)
0926 21:28:03.460026 (+ 29230us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.460038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.460077 (+ 39us) server_negotiation.cc:407] Connection header received
0926 21:28:03.460179 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.460187 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.460488 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.460722 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:03.461409 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.463014 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.465586 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.468451 (+ 2865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.468732 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.469131 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.469220 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.469519 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29034,"thread_start_us":28873,"threads_started":1}
I20260926 21:28:03.472625 651 tablet_bootstrap.cc:492] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Bootstrap starting.
I20260926 21:28:03.474526 568 tablet_service.cc:1467] Processing CreateTablet for tablet ab6784394e164d548ff6c6d3572160bc (DEFAULT_TABLE table=test-table [id=d934f64a79fa41b9af307c4dd270527a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:03.476040 568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6784394e164d548ff6c6d3572160bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:03.476338 651 tablet_bootstrap.cc:654] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:03.477488 651 log.cc:824] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:03.480744 653 tablet_bootstrap.cc:492] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Bootstrap starting.
I20260926 21:28:03.481259 651 tablet_bootstrap.cc:492] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: No bootstrap required, opened a new log
I20260926 21:28:03.481578 651 ts_tablet_manager.cc:1397] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260926 21:28:03.484678 653 tablet_bootstrap.cc:654] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:03.485987 653 log.cc:824] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:03.488999 655 tablet_bootstrap.cc:492] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Bootstrap starting.
I20260926 21:28:03.489729 653 tablet_bootstrap.cc:492] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: No bootstrap required, opened a new log
I20260926 21:28:03.490010 653 ts_tablet_manager.cc:1397] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260926 21:28:03.492709 655 tablet_bootstrap.cc:654] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:03.492663 651 raft_consensus.cc:348] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.493127 651 raft_consensus.cc:374] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:03.493228 651 raft_consensus.cc:729] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd6de8b563f342b1b8986255abba8f64, State: Initialized, Role: FOLLOWER
I20260926 21:28:03.493914 655 log.cc:824] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:03.493849 651 consensus_queue.cc:260] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.495692 651 ts_tablet_manager.cc:1428] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260926 21:28:03.498874 655 tablet_bootstrap.cc:492] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: No bootstrap required, opened a new log
I20260926 21:28:03.499174 655 ts_tablet_manager.cc:1397] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 21:28:03.500269 653 raft_consensus.cc:348] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.500775 653 raft_consensus.cc:374] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:03.500877 653 raft_consensus.cc:729] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0eb1591bd744a780630a77663ff606, State: Initialized, Role: FOLLOWER
I20260926 21:28:03.501513 653 consensus_queue.cc:260] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.502748 501 heartbeater.cc:503] Master 127.14.142.62:41383 was elected leader, sending a full tablet report...
I20260926 21:28:03.503445 653 ts_tablet_manager.cc:1428] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260926 21:28:03.508405 655 raft_consensus.cc:348] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.508956 655 raft_consensus.cc:374] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:03.509047 655 raft_consensus.cc:729] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c43e58d835594152825a6dc8c7a0d372, State: Initialized, Role: FOLLOWER
I20260926 21:28:03.509552 655 consensus_queue.cc:260] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.510550 633 heartbeater.cc:503] Master 127.14.142.62:41383 was elected leader, sending a full tablet report...
I20260926 21:28:03.511199 655 ts_tablet_manager.cc:1428] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Time spent starting tablet: real 0.012s user 0.013s sys 0.001s
W20260926 21:28:03.592103 370 tablet.cc:2369] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:28:03.608093 502 tablet.cc:2369] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:28:03.619299 634 tablet.cc:2369] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:03.797789 659 raft_consensus.cc:482] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:03.798241 659 raft_consensus.cc:504] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.800729 659 leader_election.cc:288] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd6de8b563f342b1b8986255abba8f64 (127.14.142.1:37837), cd0eb1591bd744a780630a77663ff606 (127.14.142.2:39143)
I20260926 21:28:03.807165 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.801381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:55357 (local address 127.14.142.2:39143)
0926 21:28:03.801615 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.801622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.801643 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:28:03.801732 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.801739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.801823 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.801946 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:03.802390 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.803155 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.805091 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.805914 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.806124 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.806202 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.806715 (+ 513us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.806852 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 21:28:03.807183 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.800859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:39143 (local address 127.14.142.3:55357)
0926 21:28:03.801242 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:03.801262 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:03.801393 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:03.801994 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:03.801999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:03.802029 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:03.802232 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.802240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.803323 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.803329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.804917 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.804930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.806125 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.806132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.806328 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.806354 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 21:28:03.806687 (+ 333us) client_negotiation.cc:240] Negotiation successful
0926 21:28:03.806853 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260926 21:28:03.807277 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.801088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:59109 (local address 127.14.142.1:37837)
0926 21:28:03.801348 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.801357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.801992 (+ 635us) server_negotiation.cc:407] Connection header received
0926 21:28:03.802258 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.802263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.802332 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.802441 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:03.802953 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.803683 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.805545 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.806315 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.806519 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.806586 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.806910 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.807034 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260926 21:28:03.807370 660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.801325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.14.142.3:59109)
0926 21:28:03.801884 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:03.801902 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:03.802103 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:03.802446 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:03.802450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:03.802473 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:03.802786 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.802794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.803827 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.803832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.805337 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:03.805348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.806522 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.806528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:03.806753 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.806776 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 21:28:03.806881 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 21:28:03.807020 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"spinlock_wait_cycles":23424,"thread_start_us":410,"threads_started":1}
I20260926 21:28:03.808451 456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6784394e164d548ff6c6d3572160bc" candidate_uuid: "c43e58d835594152825a6dc8c7a0d372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0eb1591bd744a780630a77663ff606" is_pre_election: true
I20260926 21:28:03.808451 324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6784394e164d548ff6c6d3572160bc" candidate_uuid: "c43e58d835594152825a6dc8c7a0d372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd6de8b563f342b1b8986255abba8f64" is_pre_election: true
I20260926 21:28:03.809087 324 raft_consensus.cc:2454] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43e58d835594152825a6dc8c7a0d372 in term 0.
I20260926 21:28:03.809087 456 raft_consensus.cc:2454] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c43e58d835594152825a6dc8c7a0d372 in term 0.
I20260926 21:28:03.810029 522 leader_election.cc:302] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: c43e58d835594152825a6dc8c7a0d372, cd0eb1591bd744a780630a77663ff606, fd6de8b563f342b1b8986255abba8f64; no voters:
I20260926 21:28:03.810495 659 raft_consensus.cc:2781] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:28:03.810662 659 raft_consensus.cc:482] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:28:03.810778 659 raft_consensus.cc:3037] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:03.813534 659 raft_consensus.cc:504] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.814319 659 leader_election.cc:288] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [CANDIDATE]: Term 1 election: Requested vote from peers fd6de8b563f342b1b8986255abba8f64 (127.14.142.1:37837), cd0eb1591bd744a780630a77663ff606 (127.14.142.2:39143)
I20260926 21:28:03.814823 324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6784394e164d548ff6c6d3572160bc" candidate_uuid: "c43e58d835594152825a6dc8c7a0d372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd6de8b563f342b1b8986255abba8f64"
I20260926 21:28:03.814921 456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6784394e164d548ff6c6d3572160bc" candidate_uuid: "c43e58d835594152825a6dc8c7a0d372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd0eb1591bd744a780630a77663ff606"
I20260926 21:28:03.815047 324 raft_consensus.cc:3037] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:03.815207 456 raft_consensus.cc:3037] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:03.817523 324 raft_consensus.cc:2454] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43e58d835594152825a6dc8c7a0d372 in term 1.
I20260926 21:28:03.817873 456 raft_consensus.cc:2454] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c43e58d835594152825a6dc8c7a0d372 in term 1.
I20260926 21:28:03.818023 522 leader_election.cc:302] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c43e58d835594152825a6dc8c7a0d372, fd6de8b563f342b1b8986255abba8f64; no voters:
I20260926 21:28:03.818387 659 raft_consensus.cc:2781] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:03.818938 659 raft_consensus.cc:686] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 1 LEADER]: Becoming Leader. State: Replica: c43e58d835594152825a6dc8c7a0d372, State: Running, Role: LEADER
I20260926 21:28:03.819473 659 consensus_queue.cc:237] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } }
I20260926 21:28:03.825273 32649 catalog_manager.cc:5495] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 reported cstate change: term changed from 0 to 1, leader changed from <none> to c43e58d835594152825a6dc8c7a0d372 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "c43e58d835594152825a6dc8c7a0d372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c43e58d835594152825a6dc8c7a0d372" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 33213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 } health_report { overall_health: UNKNOWN } } }
I20260926 21:28:03.898412 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.892537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.14.142.3:33213)
0926 21:28:03.892839 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.892844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.892914 (+ 70us) server_negotiation.cc:407] Connection header received
0926 21:28:03.893122 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.893126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.893215 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.893356 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:28:03.893760 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.894461 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.895948 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.896350 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.896451 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.897106 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:28:03.897677 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:28:03.897894 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.897939 (+ 45us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.898052 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 21:28:03.907326 456 raft_consensus.cc:1261] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 1 FOLLOWER]: Refusing update from remote peer c43e58d835594152825a6dc8c7a0d372: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:03.907718 324 raft_consensus.cc:1261] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 1 FOLLOWER]: Refusing update from remote peer c43e58d835594152825a6dc8c7a0d372: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:03.908504 661 consensus_queue.cc:1035] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd0eb1591bd744a780630a77663ff606" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 39143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:03.908962 659 consensus_queue.cc:1035] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd6de8b563f342b1b8986255abba8f64" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 37837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:03.938871 32650 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:03.939177 32650 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:03.943360 32650 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f16ad9a638134a0ea3186027eb290792: Sending DeleteTablet for 3 replicas of tablet ab6784394e164d548ff6c6d3572160bc
I20260926 21:28:03.944911 304 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6784394e164d548ff6c6d3572160bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:03 UTC) from {username='slave'} at 127.0.0.1:50910
I20260926 21:28:03.944911 436 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6784394e164d548ff6c6d3572160bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:03 UTC) from {username='slave'} at 127.0.0.1:54814
I20260926 21:28:03.945181 568 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6784394e164d548ff6c6d3572160bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:03 UTC) from {username='slave'} at 127.0.0.1:38728
I20260926 21:28:03.947121 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 32709
I20260926 21:28:03.956521 676 tablet_replica.cc:330] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: stopping tablet replica
I20260926 21:28:03.956954 674 tablet_replica.cc:330] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: stopping tablet replica
I20260926 21:28:03.957521 674 raft_consensus.cc:2229] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:03.958243 674 raft_consensus.cc:2258] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:03.959218 675 tablet_replica.cc:330] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: stopping tablet replica
I20260926 21:28:03.959594 675 raft_consensus.cc:2229] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:03.960044 675 raft_consensus.cc:2258] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:03.961479 676 raft_consensus.cc:2229] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:28:03.962280 674 ts_tablet_manager.cc:1905] T ab6784394e164d548ff6c6d3572160bc P fd6de8b563f342b1b8986255abba8f64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:03.962265 676 raft_consensus.cc:2258] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:03.965503 676 ts_tablet_manager.cc:1905] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:03.965986 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:03.949754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50920 (local address 127.14.142.1:37837)
0926 21:28:03.950022 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:03.950032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:03.955823 (+ 5791us) server_negotiation.cc:407] Connection header received
0926 21:28:03.956967 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:03.956974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:03.957129 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:03.957241 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:03.957895 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.959135 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.961227 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:03.961888 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:03.962030 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:03.964109 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:03.964124 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:03.964143 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:03.964947 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:03.965024 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:03.965048 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:03.965157 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:03.965361 (+ 204us) server_negotiation.cc:299] Negotiation successful
0926 21:28:03.965488 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 21:28:03.968698 364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:03.970719 675 ts_tablet_manager.cc:1905] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:03.974251 676 ts_tablet_manager.cc:1918] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:03.974459 676 log.cc:1196] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/ab6784394e164d548ff6c6d3572160bc
I20260926 21:28:03.975060 676 ts_tablet_manager.cc:1939] T ab6784394e164d548ff6c6d3572160bc P c43e58d835594152825a6dc8c7a0d372: Deleting consensus metadata
I20260926 21:28:03.977039 32636 catalog_manager.cc:4841] TS c43e58d835594152825a6dc8c7a0d372 (127.14.142.3:33213): tablet ab6784394e164d548ff6c6d3572160bc (table test-table [id=d934f64a79fa41b9af307c4dd270527a]) successfully deleted
I20260926 21:28:03.980124 675 ts_tablet_manager.cc:1918] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:03.980391 675 log.cc:1196] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/ab6784394e164d548ff6c6d3572160bc
I20260926 21:28:03.980959 675 ts_tablet_manager.cc:1939] T ab6784394e164d548ff6c6d3572160bc P cd0eb1591bd744a780630a77663ff606: Deleting consensus metadata
I20260926 21:28:03.982738 32634 catalog_manager.cc:4841] TS cd0eb1591bd744a780630a77663ff606 (127.14.142.2:39143): tablet ab6784394e164d548ff6c6d3572160bc (table test-table [id=d934f64a79fa41b9af307c4dd270527a]) successfully deleted
I20260926 21:28:04.133821 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 32709
W20260926 21:28:04.154477 32636 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.142.1:37837 (error 108)
I20260926 21:28:04.154742 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 373
I20260926 21:28:04.157570 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.156026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50936)
0926 21:28:04.157181 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.157351 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1029,"thread_start_us":892,"threads_started":1}
W20260926 21:28:04.157868 32636 catalog_manager.cc:4584] TS fd6de8b563f342b1b8986255abba8f64 (127.14.142.1:37837): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab6784394e164d548ff6c6d3572160bc: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
I20260926 21:28:04.163160 648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.155675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.14.142.2:39143)
0926 21:28:04.155889 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:04.155896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:04.155944 (+ 48us) server_negotiation.cc:407] Connection header received
0926 21:28:04.156019 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:04.156024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:04.156142 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:04.156298 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:04.156899 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:04.157804 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:04.159314 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:04.160002 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:04.160158 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:04.161589 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:04.161599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:04.161615 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:04.162337 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:04.162391 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:04.162414 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:04.162552 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:04.162673 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 21:28:04.162793 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 21:28:04.163929 496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:04.211946 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.211133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50946)
0926 21:28:04.211485 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.211689 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260926 21:28:04.213537 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.212946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50948)
0926 21:28:04.213174 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.213305 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 21:28:04.256526 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 373
I20260926 21:28:04.274547 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.273889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50956)
0926 21:28:04.274213 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.274326 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260926 21:28:04.277231 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.275284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50966)
0926 21:28:04.276877 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.276986 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1460,"threads_started":1}
I20260926 21:28:04.278224 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 505
I20260926 21:28:04.286295 649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.279061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.14.142.3:33213)
0926 21:28:04.279293 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:04.279302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:04.279322 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:28:04.279389 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:04.279394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:04.279479 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:04.279612 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:04.280020 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:04.280961 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:04.282261 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:04.282713 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:04.282878 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:04.284415 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:04.284427 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:04.284447 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:04.285329 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:04.285393 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:04.285419 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:04.285563 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:04.285728 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 21:28:04.285864 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 21:28:04.287333 628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:04.358103 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.357402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50970)
0926 21:28:04.357697 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.357886 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260926 21:28:04.359316 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.358856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50974)
0926 21:28:04.359086 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.359172 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260926 21:28:04.408245 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 505
I20260926 21:28:04.429004 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 32619
I20260926 21:28:04.430246 32680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:04.543154 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 32619
I20260926 21:28:04.543258 682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:04.542749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:37837 (local address 127.0.0.1:50988)
0926 21:28:04.542937 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:04.543074 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:37837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
2026-09-26T21:28:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3384 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:04 dist-test-slave-4d5t krb5kdc[687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:04 dist-test-slave-4d5t krb5kdc[687](info): set up 2 sockets
Sep 26 21:28:04 dist-test-slave-4d5t krb5kdc[687](info): commencing operation
krb5kdc: starting...
W20260926 21:28:05.717753 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:05 dist-test-slave-4d5t krb5kdc[687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:06.005578 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:43027
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:06.364370 700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:06.365125 700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:06.365340 700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:06.373236 700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:06.373406 700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:06.373528 700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:06.373629 700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:06.373719 700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:06.384102 700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:43027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:06.385588 700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:06.386940 700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:06.399290 708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:06.399334 705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:06.399282 706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:06.400723 700 server_base.cc:1034] running on GCE node
I20260926 21:28:06.401563 700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:06.403018 700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:06.404284 700 hybrid_clock.cc:648] HybridClock initialized: now 1790458086404242 us; error 56 us; skew 500 ppm
I20260926 21:28:06.404754 700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:06 dist-test-slave-4d5t krb5kdc[687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458086, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:06.417826 700 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:06.419821 700 webserver.cc:460] Webserver started at http://127.14.142.62:34775/ using document root <none> and password file <none>
I20260926 21:28:06.420682 700 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:06.420846 700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:06.421176 700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:06.424448 700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "e6662e71a549431f93c29025ad2e2708"
format_stamp: "Formatted at 2026-09-26 21:28:06 on dist-test-slave-4d5t"
server_key: "b6dcea961d5e27ea26c05dc69b9b3aff"
server_key_iv: "b54164b47fd514db7c0d77f4099cf7d0"
server_key_version: "encryptionkey@0"
I20260926 21:28:06.425271 700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "e6662e71a549431f93c29025ad2e2708"
format_stamp: "Formatted at 2026-09-26 21:28:06 on dist-test-slave-4d5t"
server_key: "b6dcea961d5e27ea26c05dc69b9b3aff"
server_key_iv: "b54164b47fd514db7c0d77f4099cf7d0"
server_key_version: "encryptionkey@0"
I20260926 21:28:06.431828 700 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:28:06.437026 715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:06.438599 700 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 21:28:06.438824 700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "e6662e71a549431f93c29025ad2e2708"
format_stamp: "Formatted at 2026-09-26 21:28:06 on dist-test-slave-4d5t"
server_key: "b6dcea961d5e27ea26c05dc69b9b3aff"
server_key_iv: "b54164b47fd514db7c0d77f4099cf7d0"
server_key_version: "encryptionkey@0"
I20260926 21:28:06.439064 700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:06.452369 700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:06.456944 700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:06.457244 700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:06.479003 700 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:43027
I20260926 21:28:06.479024 766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:43027 every 8 connection(s)
I20260926 21:28:06.480774 700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:06.485399 767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:06.486008 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 700
I20260926 21:28:06.486244 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:06.486879 14904 external_mini_cluster.cc:1428] Setting key 9cf6c0bc37740dc00cea77ecb1b110d5
I20260926 21:28:06.499037 767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708: Bootstrap starting.
I20260926 21:28:06.502854 767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708: Neither blocks nor log segments found. Creating new log.
Sep 26 21:28:06 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:06.504199 767 log.cc:824] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:06.507560 767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708: No bootstrap required, opened a new log
I20260926 21:28:06.513497 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:06.489367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.14.142.62:43027)
0926 21:28:06.490453 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:06.490467 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:06.490512 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:28:06.492618 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:06.492657 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:06.493127 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:06.493585 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:06.495306 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:06.496862 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:06.498417 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:06.499239 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:06.499479 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:06.505148 (+ 5669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:06.505157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:06.505175 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:06.505235 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:06.508628 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:06.509386 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:06.509402 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:06.509408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:06.509591 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:06.509997 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:06.510002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:06.510005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:06.510719 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:06.511137 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:06.511691 (+ 554us) server_negotiation.cc:299] Negotiation successful
0926 21:28:06.512219 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":430,"threads_started":1}
I20260926 21:28:06.515728 767 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6662e71a549431f93c29025ad2e2708" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43027 } }
I20260926 21:28:06.516121 767 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:06.516253 767 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6662e71a549431f93c29025ad2e2708, State: Initialized, Role: FOLLOWER
I20260926 21:28:06.516958 767 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6662e71a549431f93c29025ad2e2708" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43027 } }
I20260926 21:28:06.517194 767 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:06.517364 767 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:06.517550 767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:06.519965 767 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6662e71a549431f93c29025ad2e2708" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43027 } }
I20260926 21:28:06.520668 767 leader_election.cc:302] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6662e71a549431f93c29025ad2e2708; no voters:
I20260926 21:28:06.521333 767 leader_election.cc:288] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:06.521528 772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:06.522766 772 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [term 1 LEADER]: Becoming Leader. State: Replica: e6662e71a549431f93c29025ad2e2708, State: Running, Role: LEADER
I20260926 21:28:06.523929 767 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:06.523829 772 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6662e71a549431f93c29025ad2e2708" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43027 } }
I20260926 21:28:06.528769 774 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6662e71a549431f93c29025ad2e2708. Latest consensus state: current_term: 1 leader_uuid: "e6662e71a549431f93c29025ad2e2708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6662e71a549431f93c29025ad2e2708" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43027 } } }
I20260926 21:28:06.528740 773 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6662e71a549431f93c29025ad2e2708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6662e71a549431f93c29025ad2e2708" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 43027 } } }
I20260926 21:28:06.529315 774 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:06.529321 773 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:06.530768 782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:06.538223 782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:06.549913 782 catalog_manager.cc:1347] Generated new cluster ID: b09f4063ee334f848a52353244c16458
I20260926 21:28:06.550077 782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:06.569270 782 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:06.571859 782 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:06.580139 782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708: Generated new TSK 0
I20260926 21:28:06.581171 782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:28:06.661007 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:43027
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:07.024484 793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:07.024925 793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:07.025085 793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:07.034224 793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:07.034412 793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:07.034602 793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:28:07.045290 793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:07.046741 793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:07.048233 793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:07.064704 798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:07.064636 801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:07.064615 799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:07.065672 793 server_base.cc:1034] running on GCE node
I20260926 21:28:07.066462 793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:07.067468 793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:07.068715 793 hybrid_clock.cc:648] HybridClock initialized: now 1790458087068633 us; error 65 us; skew 500 ppm
I20260926 21:28:07.069025 793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:07 dist-test-slave-4d5t krb5kdc[687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458087, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:07.072374 793 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:28:07.074072 793 webserver.cc:460] Webserver started at http://127.14.142.1:42199/ using document root <none> and password file <none>
I20260926 21:28:07.074829 793 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:07.074990 793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:07.075330 793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:07.078578 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "5acb5cbb42964a5f9ee5c90fa339393d"
format_stamp: "Formatted at 2026-09-26 21:28:07 on dist-test-slave-4d5t"
server_key: "7970ad125e857629a501a2e986f77f58"
server_key_iv: "99824ccd20274e89e308b05bb82fa884"
server_key_version: "encryptionkey@0"
I20260926 21:28:07.079522 793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "5acb5cbb42964a5f9ee5c90fa339393d"
format_stamp: "Formatted at 2026-09-26 21:28:07 on dist-test-slave-4d5t"
server_key: "7970ad125e857629a501a2e986f77f58"
server_key_iv: "99824ccd20274e89e308b05bb82fa884"
server_key_version: "encryptionkey@0"
I20260926 21:28:07.085427 793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:28:07.089340 808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:07.090960 793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:28:07.091179 793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "5acb5cbb42964a5f9ee5c90fa339393d"
format_stamp: "Formatted at 2026-09-26 21:28:07 on dist-test-slave-4d5t"
server_key: "7970ad125e857629a501a2e986f77f58"
server_key_iv: "99824ccd20274e89e308b05bb82fa884"
server_key_version: "encryptionkey@0"
I20260926 21:28:07.091410 793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:07.124290 793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:07.129506 793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:07.129869 793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:07.131099 793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:07.132843 793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:07.132963 793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:07.133137 793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:07.133219 793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:07.175426 793 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:38491
I20260926 21:28:07.175540 920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:38491 every 8 connection(s)
I20260926 21:28:07.177042 793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:28:07.183894 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 793
I20260926 21:28:07.184170 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:28:07.184801 14904 external_mini_cluster.cc:1428] Setting key 535a873874af5c038f2b88c3acdd5572
Sep 26 21:28:07 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458087, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:07.203123 923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:07.180056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:37313 (local address 127.14.142.62:43027)
0926 21:28:07.180668 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:07.180673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:07.181540 (+ 867us) server_negotiation.cc:407] Connection header received
0926 21:28:07.183744 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:07.183759 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:07.183835 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:07.183954 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:07.186050 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.186887 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.189510 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.190004 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.190132 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:07.194852 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:07.194863 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:07.194871 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:07.194924 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:07.199378 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:07.200331 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:07.200338 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:07.200342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:07.200491 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:07.201074 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:07.201080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:07.201084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:07.201400 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:07.201594 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:07.202408 (+ 814us) server_negotiation.cc:299] Negotiation successful
0926 21:28:07.202704 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":275,"threads_started":1}
I20260926 21:28:07.204267 924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:07.180663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:43027 (local address 127.14.142.1:37313)
0926 21:28:07.181381 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:07.181419 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:07.183349 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:07.184267 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:07.184280 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:07.185116 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:07.185852 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:07.185871 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.187087 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.187091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:07.189293 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:07.189314 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.190274 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.190283 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:07.190664 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:07.191582 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:07.191617 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:07.194495 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:07.199590 (+ 5095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:07.199603 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:07.199627 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:07.200158 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:07.200683 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:07.200689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:07.200692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:07.200931 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:07.201552 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:07.201562 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:07.201968 (+ 406us) client_negotiation.cc:769] Sending connection context
0926 21:28:07.202279 (+ 311us) client_negotiation.cc:240] Negotiation successful
0926 21:28:07.202898 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":296,"threads_started":1}
I20260926 21:28:07.206317 921 heartbeater.cc:347] Connected to a master server at 127.14.142.62:43027
I20260926 21:28:07.206910 921 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:07.207953 921 heartbeater.cc:511] Master 127.14.142.62:43027 requested a full tablet report, sending...
I20260926 21:28:07.211014 732 ts_manager.cc:194] Registered new tserver with Master: 5acb5cbb42964a5f9ee5c90fa339393d (127.14.142.1:38491)
I20260926 21:28:07.212718 732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:37313
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:28:07.253507 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:43027
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:07.611217 927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:07.611526 927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:07.611624 927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:07.619139 927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:07.619279 927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:07.619398 927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:28:07.630040 927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:07.631399 927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:07.632865 927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:07.647449 933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:07.647523 932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:07.647526 935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:07.648432 927 server_base.cc:1034] running on GCE node
I20260926 21:28:07.649196 927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:07.650271 927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:07.651544 927 hybrid_clock.cc:648] HybridClock initialized: now 1790458087651492 us; error 60 us; skew 500 ppm
I20260926 21:28:07.652372 927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:07 dist-test-slave-4d5t krb5kdc[687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458087, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:07.655732 927 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:28:07.657573 927 webserver.cc:460] Webserver started at http://127.14.142.2:34805/ using document root <none> and password file <none>
I20260926 21:28:07.658308 927 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:07.658422 927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:07.658718 927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:07.661870 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "b87ae68a890744febfdafe6add98b69e"
format_stamp: "Formatted at 2026-09-26 21:28:07 on dist-test-slave-4d5t"
server_key: "bb5022ed6678632c5107fee4b5b91208"
server_key_iv: "7da31b0019032da40807067a7fc1d6c7"
server_key_version: "encryptionkey@0"
I20260926 21:28:07.662760 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "b87ae68a890744febfdafe6add98b69e"
format_stamp: "Formatted at 2026-09-26 21:28:07 on dist-test-slave-4d5t"
server_key: "bb5022ed6678632c5107fee4b5b91208"
server_key_iv: "7da31b0019032da40807067a7fc1d6c7"
server_key_version: "encryptionkey@0"
I20260926 21:28:07.668749 927 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 21:28:07.672930 942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:07.674840 927 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:28:07.675050 927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "b87ae68a890744febfdafe6add98b69e"
format_stamp: "Formatted at 2026-09-26 21:28:07 on dist-test-slave-4d5t"
server_key: "bb5022ed6678632c5107fee4b5b91208"
server_key_iv: "7da31b0019032da40807067a7fc1d6c7"
server_key_version: "encryptionkey@0"
I20260926 21:28:07.675239 927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:07.704613 927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:07.709585 927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:07.709882 927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:07.710994 927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:07.712677 927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:07.712761 927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:07.712870 927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:07.712931 927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:07.754035 927 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:35425
I20260926 21:28:07.754151 1054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:35425 every 8 connection(s)
I20260926 21:28:07.755487 927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:28:07.764478 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 927
I20260926 21:28:07.764703 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:28:07.765162 14904 external_mini_cluster.cc:1428] Setting key 917a08c74c5249067b2dd4ce9f933822
Sep 26 21:28:07 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458087, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:07.779810 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:07.758178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:36671 (local address 127.14.142.62:43027)
0926 21:28:07.758669 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:07.758673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:07.760048 (+ 1375us) server_negotiation.cc:407] Connection header received
0926 21:28:07.762429 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:07.762436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:07.762546 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:07.762674 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:07.764411 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.765531 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.767578 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.768004 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.768113 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:07.773049 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:07.773059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:07.773065 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:07.773118 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:07.776050 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:07.776954 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:07.776962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:07.776967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:07.777151 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:07.777671 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:07.777679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:07.777685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:07.778097 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:07.778252 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:07.779018 (+ 766us) server_negotiation.cc:299] Negotiation successful
0926 21:28:07.779286 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":260,"threads_started":1}
I20260926 21:28:07.780790 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:07.758687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:43027 (local address 127.14.142.2:36671)
0926 21:28:07.759782 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:07.759846 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:07.762075 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:07.762931 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:07.762940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:07.763468 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:07.764169 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:07.764198 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.765715 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.765719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:07.767392 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:07.767406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:07.768255 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:07.768264 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:07.768664 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:07.769482 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:07.769517 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:07.772698 (+ 3181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:07.776238 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:07.776251 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:07.776278 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:07.776830 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:07.777304 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:07.777308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:07.777310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:07.777532 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:07.778239 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:07.778245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:07.778613 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 21:28:07.778856 (+ 243us) client_negotiation.cc:240] Negotiation successful
0926 21:28:07.779443 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":456,"threads_started":1}
I20260926 21:28:07.782784 1055 heartbeater.cc:347] Connected to a master server at 127.14.142.62:43027
I20260926 21:28:07.783210 1055 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:07.784221 1055 heartbeater.cc:511] Master 127.14.142.62:43027 requested a full tablet report, sending...
I20260926 21:28:07.786294 732 ts_manager.cc:194] Registered new tserver with Master: b87ae68a890744febfdafe6add98b69e (127.14.142.2:35425)
I20260926 21:28:07.787706 732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:36671
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:28:07.828397 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:43027
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:08.173945 1061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:08.174237 1061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:08.174410 1061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:08.182227 1061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:08.182406 1061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:08.182586 1061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:28:08.192806 1061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:43027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:08.194149 1061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:08.195518 1061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:08.209203 1066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:08.209113 1067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:08.209311 1069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:08.210299 1061 server_base.cc:1034] running on GCE node
I20260926 21:28:08.210969 1061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:08.211977 1061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:08.213330 1061 hybrid_clock.cc:648] HybridClock initialized: now 1790458088213224 us; error 65 us; skew 500 ppm
I20260926 21:28:08.213938 1061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:08.216284 921 heartbeater.cc:503] Master 127.14.142.62:43027 was elected leader, sending a full tablet report...
Sep 26 21:28:08 dist-test-slave-4d5t krb5kdc[687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458088, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:08.217394 1061 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:28:08.219151 1061 webserver.cc:460] Webserver started at http://127.14.142.3:36415/ using document root <none> and password file <none>
I20260926 21:28:08.219877 1061 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:08.220036 1061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:08.220364 1061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:08.223461 1061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "f78e95de50034399b1d080da8800b838"
format_stamp: "Formatted at 2026-09-26 21:28:08 on dist-test-slave-4d5t"
server_key: "5113c47e2f4490a4d02dd2aa3c8d61b4"
server_key_iv: "d31af4262203e28ae0c14b91eaff0fe2"
server_key_version: "encryptionkey@0"
I20260926 21:28:08.224326 1061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "f78e95de50034399b1d080da8800b838"
format_stamp: "Formatted at 2026-09-26 21:28:08 on dist-test-slave-4d5t"
server_key: "5113c47e2f4490a4d02dd2aa3c8d61b4"
server_key_iv: "d31af4262203e28ae0c14b91eaff0fe2"
server_key_version: "encryptionkey@0"
I20260926 21:28:08.230008 1061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 21:28:08.234046 1076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:08.236027 1061 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:28:08.236263 1061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "f78e95de50034399b1d080da8800b838"
format_stamp: "Formatted at 2026-09-26 21:28:08 on dist-test-slave-4d5t"
server_key: "5113c47e2f4490a4d02dd2aa3c8d61b4"
server_key_iv: "d31af4262203e28ae0c14b91eaff0fe2"
server_key_version: "encryptionkey@0"
I20260926 21:28:08.236519 1061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:08.268584 1061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:08.273523 1061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:08.273931 1061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:08.275085 1061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:08.276723 1061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:08.276841 1061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:08.277004 1061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:08.277092 1061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:08.315986 1061 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:39121
I20260926 21:28:08.316084 1188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:39121 every 8 connection(s)
I20260926 21:28:08.317451 1061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:28:08.327260 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1061
I20260926 21:28:08.327461 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:28:08.328020 14904 external_mini_cluster.cc:1428] Setting key 7b39ee54056eba8efa07f88016a74b9e
Sep 26 21:28:08 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458088, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:08.341115 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.320226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:39641 (local address 127.14.142.62:43027)
0926 21:28:08.320851 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.320856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.321635 (+ 779us) server_negotiation.cc:407] Connection header received
0926 21:28:08.323722 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.323728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.323845 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.323979 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:08.325894 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.326826 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.328895 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.329435 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.329585 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.333699 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:08.333704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:08.333709 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:08.333754 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:08.337206 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:08.338319 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:08.338324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:08.338326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:08.338474 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:08.339133 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:08.339138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:08.339140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:08.339473 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:08.339623 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.340487 (+ 864us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.340752 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":366,"threads_started":1}
I20260926 21:28:08.342676 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.320722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:43027 (local address 127.14.142.3:39641)
0926 21:28:08.321452 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:08.321502 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:08.323413 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:08.324335 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:08.324344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:08.324997 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:08.325711 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.325732 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.327025 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.327030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.328695 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.328711 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.329687 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.329695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.330027 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.330895 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:08.330933 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:08.333439 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:08.337477 (+ 4038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:08.337489 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:08.337513 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:08.338139 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:08.338654 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:08.338660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:08.338663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:08.338949 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:08.339648 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:08.339655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:08.340062 (+ 407us) client_negotiation.cc:769] Sending connection context
0926 21:28:08.340339 (+ 277us) client_negotiation.cc:240] Negotiation successful
0926 21:28:08.340917 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":297,"threads_started":1}
I20260926 21:28:08.344301 1189 heartbeater.cc:347] Connected to a master server at 127.14.142.62:43027
I20260926 21:28:08.344683 1189 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:08.345427 1189 heartbeater.cc:511] Master 127.14.142.62:43027 requested a full tablet report, sending...
I20260926 21:28:08.346928 732 ts_manager.cc:194] Registered new tserver with Master: f78e95de50034399b1d080da8800b838 (127.14.142.3:39121)
I20260926 21:28:08.347724 732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:39641
I20260926 21:28:08.356617 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:28:08.374572 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.360969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.14.142.62:43027)
0926 21:28:08.361149 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.361153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.361552 (+ 399us) server_negotiation.cc:407] Connection header received
0926 21:28:08.361674 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.361678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.361742 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.361833 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:08.363290 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.364048 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.365533 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.365920 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.366020 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.368737 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:08.368742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:08.368745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:08.368779 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:08.371585 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:08.372474 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:08.372479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:08.372481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:08.372681 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:08.373179 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:08.373182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:08.373184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:08.373487 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:08.373700 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.374146 (+ 446us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.374302 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:28:08.379245 732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:28:08.386137 732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:28:08.413772 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.401009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.14.142.3:39121)
0926 21:28:08.401918 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.401928 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.401961 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:28:08.402056 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.402062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.402286 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.403916 (+ 1630us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:08.404264 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.406394 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.410548 (+ 4154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.412361 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.412838 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.413065 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.413230 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.413394 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":935,"threads_started":1}
I20260926 21:28:08.413846 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.400334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:39121 (local address 127.0.0.1:49020)
0926 21:28:08.401103 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:08.401150 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:08.401344 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:08.402567 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:08.402578 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:08.402624 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:08.403222 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.403236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.406611 (+ 3375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.406621 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.410242 (+ 3621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.410265 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.412611 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.412620 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.412930 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.413011 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 21:28:08.413184 (+ 173us) client_negotiation.cc:240] Negotiation successful
0926 21:28:08.413392 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":72,"spinlock_wait_cycles":6912,"thread_start_us":660,"threads_started":1}
I20260926 21:28:08.415318 1204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.399524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:35425 (local address 127.0.0.1:51332)
0926 21:28:08.401103 (+ 1579us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:08.401150 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:08.401343 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:08.402428 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:08.402439 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:08.402475 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:08.403613 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.403626 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.406210 (+ 2584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.406218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.409655 (+ 3437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.409683 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.412537 (+ 2854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.412547 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.414528 (+ 1981us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.414566 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:28:08.414687 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 21:28:08.414848 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1298,"mutex_wait_us":15,"spinlock_wait_cycles":26240,"thread_start_us":1784,"threads_started":1}
I20260926 21:28:08.415309 1203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.399740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.14.142.2:35425)
0926 21:28:08.400613 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.400627 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.401543 (+ 916us) server_negotiation.cc:407] Connection header received
0926 21:28:08.401662 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.401668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.401920 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.402103 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:08.403830 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.405954 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.409955 (+ 4001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.411997 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.412119 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.412331 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.414755 (+ 2424us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.414946 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":437,"threads_started":1}
I20260926 21:28:08.418260 1124 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9c08c423844a928cee58ec07c20b76 (DEFAULT_TABLE table=test-table [id=0a9033b4569e4ea6b932ec32c920194e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:08.419185 990 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9c08c423844a928cee58ec07c20b76 (DEFAULT_TABLE table=test-table [id=0a9033b4569e4ea6b932ec32c920194e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:08.419971 1124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9c08c423844a928cee58ec07c20b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:08.420450 990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9c08c423844a928cee58ec07c20b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:08.421002 1206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.401831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58696)
0926 21:28:08.412160 (+ 10329us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:08.412182 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:08.412307 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:08.413782 (+ 1475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:08.413789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:08.413830 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:08.414257 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.414276 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.416225 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.416232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.418383 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.418402 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.420011 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.420019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.420227 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.420266 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 21:28:08.420416 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 21:28:08.420570 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10174,"thread_start_us":10568,"threads_started":1}
I20260926 21:28:08.421032 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.402086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.14.142.1:38491)
0926 21:28:08.413099 (+ 11013us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.413110 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.413140 (+ 30us) server_negotiation.cc:407] Connection header received
0926 21:28:08.413250 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.413258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.413540 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.413781 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:08.414477 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.416051 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.418556 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.419725 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.419940 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.420195 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.420473 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.420675 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10820,"thread_start_us":10984,"threads_started":1}
I20260926 21:28:08.424846 856 tablet_service.cc:1467] Processing CreateTablet for tablet 8f9c08c423844a928cee58ec07c20b76 (DEFAULT_TABLE table=test-table [id=0a9033b4569e4ea6b932ec32c920194e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:08.426123 856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f9c08c423844a928cee58ec07c20b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:08.437448 1208 tablet_bootstrap.cc:492] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Bootstrap starting.
I20260926 21:28:08.438302 1209 tablet_bootstrap.cc:492] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Bootstrap starting.
I20260926 21:28:08.442749 1208 tablet_bootstrap.cc:654] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:08.442893 1209 tablet_bootstrap.cc:654] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:08.444018 1208 log.cc:824] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:08.444475 1209 log.cc:824] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:08.447005 1208 tablet_bootstrap.cc:492] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: No bootstrap required, opened a new log
I20260926 21:28:08.447283 1208 ts_tablet_manager.cc:1397] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 21:28:08.448648 1210 tablet_bootstrap.cc:492] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Bootstrap starting.
I20260926 21:28:08.452615 1210 tablet_bootstrap.cc:654] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:08.453922 1209 tablet_bootstrap.cc:492] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: No bootstrap required, opened a new log
I20260926 21:28:08.454088 1210 log.cc:824] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:08.454275 1209 ts_tablet_manager.cc:1397] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.004s
I20260926 21:28:08.457449 1210 tablet_bootstrap.cc:492] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: No bootstrap required, opened a new log
I20260926 21:28:08.457747 1210 ts_tablet_manager.cc:1397] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.001s
I20260926 21:28:08.457515 1208 raft_consensus.cc:348] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.458011 1208 raft_consensus.cc:374] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:08.458181 1208 raft_consensus.cc:729] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f78e95de50034399b1d080da8800b838, State: Initialized, Role: FOLLOWER
I20260926 21:28:08.458842 1208 consensus_queue.cc:260] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.460116 1189 heartbeater.cc:503] Master 127.14.142.62:43027 was elected leader, sending a full tablet report...
I20260926 21:28:08.461483 1208 ts_tablet_manager.cc:1428] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260926 21:28:08.463845 1209 raft_consensus.cc:348] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.464327 1209 raft_consensus.cc:374] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:08.464457 1209 raft_consensus.cc:729] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87ae68a890744febfdafe6add98b69e, State: Initialized, Role: FOLLOWER
I20260926 21:28:08.465444 1209 consensus_queue.cc:260] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.467661 1055 heartbeater.cc:503] Master 127.14.142.62:43027 was elected leader, sending a full tablet report...
I20260926 21:28:08.467928 1210 raft_consensus.cc:348] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.468168 1209 ts_tablet_manager.cc:1428] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260926 21:28:08.468315 1210 raft_consensus.cc:374] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:08.468429 1210 raft_consensus.cc:729] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acb5cbb42964a5f9ee5c90fa339393d, State: Initialized, Role: FOLLOWER
I20260926 21:28:08.469045 1210 consensus_queue.cc:260] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.470811 1210 ts_tablet_manager.cc:1428] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
W20260926 21:28:08.507951 1056 tablet.cc:2369] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:28:08.569267 1190 tablet.cc:2369] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:28:08.680184 922 tablet.cc:2369] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:08.851019 1215 raft_consensus.cc:482] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:08.851522 1215 raft_consensus.cc:504] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.854056 1215 leader_election.cc:288] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5acb5cbb42964a5f9ee5c90fa339393d (127.14.142.1:38491), f78e95de50034399b1d080da8800b838 (127.14.142.3:39121)
I20260926 21:28:08.863282 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.854492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:50691 (local address 127.14.142.3:39121)
0926 21:28:08.854714 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.854723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.855537 (+ 814us) server_negotiation.cc:407] Connection header received
0926 21:28:08.855800 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.855807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.855884 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.855988 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:08.856962 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.858057 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.861290 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.862132 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.862381 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.862479 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.862813 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.862974 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:28:08.863348 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.854643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:39037 (local address 127.14.142.1:38491)
0926 21:28:08.854864 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.854868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.855986 (+ 1118us) server_negotiation.cc:407] Connection header received
0926 21:28:08.856207 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.856218 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.856283 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.856406 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:08.857154 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.858087 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.861252 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.862188 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.862431 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.862529 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.862887 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.863038 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:28:08.863503 1217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.854717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:39121 (local address 127.14.142.2:50691)
0926 21:28:08.855412 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:08.855429 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:08.855643 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:08.856007 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:08.856011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:08.856101 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:08.856690 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.856719 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.858236 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.858243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.861061 (+ 2818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.861077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.862427 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.862436 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.862621 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.862659 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:28:08.862774 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 21:28:08.862946 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":106,"spinlock_wait_cycles":28928,"thread_start_us":609,"threads_started":1}
I20260926 21:28:08.863590 1218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.855084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.14.142.2:39037)
0926 21:28:08.855854 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:08.855870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:08.856055 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:08.856415 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:08.856420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:08.856444 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:08.856957 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.856968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.858242 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.858250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.861048 (+ 2798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:08.861067 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.862513 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.862527 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:08.862695 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.862728 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 21:28:08.862856 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 21:28:08.863041 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":33,"thread_start_us":547,"threads_started":1}
I20260926 21:28:08.865015 1144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9c08c423844a928cee58ec07c20b76" candidate_uuid: "b87ae68a890744febfdafe6add98b69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f78e95de50034399b1d080da8800b838" is_pre_election: true
I20260926 21:28:08.865083 876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9c08c423844a928cee58ec07c20b76" candidate_uuid: "b87ae68a890744febfdafe6add98b69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" is_pre_election: true
I20260926 21:28:08.865624 1144 raft_consensus.cc:2454] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b87ae68a890744febfdafe6add98b69e in term 0.
I20260926 21:28:08.865734 876 raft_consensus.cc:2454] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b87ae68a890744febfdafe6add98b69e in term 0.
I20260926 21:28:08.866343 945 leader_election.cc:302] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b87ae68a890744febfdafe6add98b69e, f78e95de50034399b1d080da8800b838; no voters:
I20260926 21:28:08.866849 1215 raft_consensus.cc:2781] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:28:08.867053 1215 raft_consensus.cc:482] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:28:08.867265 1215 raft_consensus.cc:3037] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:08.870157 1215 raft_consensus.cc:504] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.871069 1215 leader_election.cc:288] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [CANDIDATE]: Term 1 election: Requested vote from peers 5acb5cbb42964a5f9ee5c90fa339393d (127.14.142.1:38491), f78e95de50034399b1d080da8800b838 (127.14.142.3:39121)
I20260926 21:28:08.871649 876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9c08c423844a928cee58ec07c20b76" candidate_uuid: "b87ae68a890744febfdafe6add98b69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acb5cbb42964a5f9ee5c90fa339393d"
I20260926 21:28:08.871742 1144 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f9c08c423844a928cee58ec07c20b76" candidate_uuid: "b87ae68a890744febfdafe6add98b69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f78e95de50034399b1d080da8800b838"
I20260926 21:28:08.871946 876 raft_consensus.cc:3037] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:08.872021 1144 raft_consensus.cc:3037] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:08.874317 1144 raft_consensus.cc:2454] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b87ae68a890744febfdafe6add98b69e in term 1.
I20260926 21:28:08.874742 876 raft_consensus.cc:2454] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b87ae68a890744febfdafe6add98b69e in term 1.
I20260926 21:28:08.874812 945 leader_election.cc:302] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b87ae68a890744febfdafe6add98b69e, f78e95de50034399b1d080da8800b838; no voters:
I20260926 21:28:08.875260 1215 raft_consensus.cc:2781] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:08.875748 1215 raft_consensus.cc:686] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 1 LEADER]: Becoming Leader. State: Replica: b87ae68a890744febfdafe6add98b69e, State: Running, Role: LEADER
I20260926 21:28:08.876252 1215 consensus_queue.cc:237] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } }
I20260926 21:28:08.881196 732 catalog_manager.cc:5495] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e reported cstate change: term changed from 0 to 1, leader changed from <none> to b87ae68a890744febfdafe6add98b69e (127.14.142.2). New cstate: current_term: 1 leader_uuid: "b87ae68a890744febfdafe6add98b69e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b87ae68a890744febfdafe6add98b69e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 } health_report { overall_health: UNKNOWN } } }
I20260926 21:28:08.983880 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:08.977779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.14.142.2:35425)
0926 21:28:08.978250 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:08.978255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:08.978330 (+ 75us) server_negotiation.cc:407] Connection header received
0926 21:28:08.978573 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:08.978579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:08.978684 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:08.978898 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:28:08.979350 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.980048 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.981574 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:08.981983 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:08.982131 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:08.982723 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:28:08.983232 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:28:08.983414 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:08.983569 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 21:28:08.983620 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":277,"threads_started":1}
I20260926 21:28:08.992899 876 raft_consensus.cc:1261] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 1 FOLLOWER]: Refusing update from remote peer b87ae68a890744febfdafe6add98b69e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:08.992899 1144 raft_consensus.cc:1261] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 1 FOLLOWER]: Refusing update from remote peer b87ae68a890744febfdafe6add98b69e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:08.993925 1215 consensus_queue.cc:1035] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f78e95de50034399b1d080da8800b838" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 39121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:08.994264 1219 consensus_queue.cc:1035] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5acb5cbb42964a5f9ee5c90fa339393d" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 38491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:09.009160 1227 mvcc.cc:205] Tried to move back new op lower bound from 7333716332503138304 to 7333716332040855552. Current Snapshot: MvccSnapshot[applied={T|T < 7333716332503138304}]
I20260926 21:28:09.009821 1224 mvcc.cc:205] Tried to move back new op lower bound from 7333716332503138304 to 7333716332040855552. Current Snapshot: MvccSnapshot[applied={T|T < 7333716332503138304}]
I20260926 21:28:09.018400 1229 mvcc.cc:205] Tried to move back new op lower bound from 7333716332503138304 to 7333716332040855552. Current Snapshot: MvccSnapshot[applied={T|T < 7333716332503138304}]
I20260926 21:28:09.029022 732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:09.029326 732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:09.034308 732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6662e71a549431f93c29025ad2e2708: Sending DeleteTablet for 3 replicas of tablet 8f9c08c423844a928cee58ec07c20b76
I20260926 21:28:09.035559 856 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9c08c423844a928cee58ec07c20b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:09 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:58696
I20260926 21:28:09.035821 1123 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9c08c423844a928cee58ec07c20b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:09 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:49020
I20260926 21:28:09.036000 1124 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9c08c423844a928cee58ec07c20b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:09 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:49020
I20260926 21:28:09.036643 990 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f9c08c423844a928cee58ec07c20b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:09 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:51332
I20260926 21:28:09.036767 1237 tablet_replica.cc:330] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: stopping tablet replica
I20260926 21:28:09.037201 1237 raft_consensus.cc:2229] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:09.037869 1237 raft_consensus.cc:2258] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:09.038198 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 793
I20260926 21:28:09.039999 1237 ts_tablet_manager.cc:1905] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:09.047591 1238 tablet_replica.cc:330] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: stopping tablet replica
I20260926 21:28:09.048233 1238 raft_consensus.cc:2229] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:28:09.049045 1238 raft_consensus.cc:2258] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:09.049508 1237 ts_tablet_manager.cc:1918] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:09.049727 1237 log.cc:1196] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/8f9c08c423844a928cee58ec07c20b76
I20260926 21:28:09.050361 1237 ts_tablet_manager.cc:1939] T 8f9c08c423844a928cee58ec07c20b76 P f78e95de50034399b1d080da8800b838: Deleting consensus metadata
I20260926 21:28:09.051146 1238 ts_tablet_manager.cc:1905] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:09.051796 1236 tablet_replica.cc:330] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: stopping tablet replica
I20260926 21:28:09.052359 718 catalog_manager.cc:4841] TS f78e95de50034399b1d080da8800b838 (127.14.142.3:39121): tablet 8f9c08c423844a928cee58ec07c20b76 (table test-table [id=0a9033b4569e4ea6b932ec32c920194e]) successfully deleted
W20260926 21:28:09.053429 718 catalog_manager.cc:4808] TS f78e95de50034399b1d080da8800b838 (127.14.142.3:39121): delete failed for tablet 8f9c08c423844a928cee58ec07c20b76 because the tablet was not found. No further retry: Not found: Tablet not found: 8f9c08c423844a928cee58ec07c20b76
I20260926 21:28:09.058825 1238 ts_tablet_manager.cc:1918] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:09.059481 1236 raft_consensus.cc:2229] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:09.060102 1236 raft_consensus.cc:2258] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:09.060686 1238 log.cc:1196] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/8f9c08c423844a928cee58ec07c20b76
I20260926 21:28:09.061229 1238 ts_tablet_manager.cc:1939] T 8f9c08c423844a928cee58ec07c20b76 P b87ae68a890744febfdafe6add98b69e: Deleting consensus metadata
I20260926 21:28:09.063122 1236 ts_tablet_manager.cc:1905] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:09.063234 719 catalog_manager.cc:4841] TS b87ae68a890744febfdafe6add98b69e (127.14.142.2:35425): tablet 8f9c08c423844a928cee58ec07c20b76 (table test-table [id=0a9033b4569e4ea6b932ec32c920194e]) successfully deleted
Sep 26 21:28:09 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:28:09.072280 1236 ts_tablet_manager.cc:1918] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:09.072546 1236 log.cc:1196] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/8f9c08c423844a928cee58ec07c20b76
I20260926 21:28:09.073067 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.040807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.14.142.1:38491)
0926 21:28:09.041015 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:09.041024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:09.054852 (+ 13828us) server_negotiation.cc:407] Connection header received
0926 21:28:09.055062 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:09.055067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:09.055168 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:09.055277 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:09.057217 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:09.058359 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:09.060228 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:09.061381 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:09.061524 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:09.066331 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:09.066346 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:09.066361 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:09.066409 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:09.069812 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:09.070583 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:09.070591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:09.070593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:09.070744 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:09.071143 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:09.071147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:09.071150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:09.071534 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:09.071852 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:09.072197 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 21:28:09.072680 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 21:28:09.073228 1236 ts_tablet_manager.cc:1939] T 8f9c08c423844a928cee58ec07c20b76 P 5acb5cbb42964a5f9ee5c90fa339393d: Deleting consensus metadata
I20260926 21:28:09.073793 916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:09.167125 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 793
W20260926 21:28:09.192708 719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.142.1:38491 (error 108)
I20260926 21:28:09.192950 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 927
I20260926 21:28:09.195056 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.194151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58716)
0926 21:28:09.194652 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.194802 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":237,"threads_started":1}
W20260926 21:28:09.195397 719 catalog_manager.cc:4584] TS 5acb5cbb42964a5f9ee5c90fa339393d (127.14.142.1:38491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f9c08c423844a928cee58ec07c20b76: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Sep 26 21:28:09 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:28:09.213357 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.194003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.14.142.2:35425)
0926 21:28:09.194203 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:09.194210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:09.194231 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:28:09.194481 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:09.194488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:09.194596 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:09.194691 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:09.196774 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:09.197803 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:09.199298 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:09.199780 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:09.199959 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:09.205570 (+ 5611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:09.205594 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:09.205633 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:09.205708 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:09.209595 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:09.210557 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:09.210566 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:09.210568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:09.210745 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:09.211214 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:09.211223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:09.211225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:09.211610 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:09.211874 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:09.212248 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 21:28:09.212476 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 21:28:09.214257 1050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:09.249384 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.248864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58718)
0926 21:28:09.249135 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.249235 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260926 21:28:09.250821 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.250164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58720)
0926 21:28:09.250467 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.250590 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260926 21:28:09.311715 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.311002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58736)
0926 21:28:09.311453 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.311552 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260926 21:28:09.312991 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.312457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58750)
0926 21:28:09.312757 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.312839 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 21:28:09.330999 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 927
I20260926 21:28:09.354398 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1061
Sep 26 21:28:09 dist-test-slave-4d5t krb5kdc[687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:28:09.372704 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.355342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.14.142.3:39121)
0926 21:28:09.355524 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:09.355528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:09.355559 (+ 31us) server_negotiation.cc:407] Connection header received
0926 21:28:09.355636 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:09.355640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:09.355728 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:09.355817 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:09.357597 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:09.358624 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:09.360190 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:09.360816 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:09.360928 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:09.366387 (+ 5459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:09.366404 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:09.366421 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:09.366477 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:09.369634 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:09.370394 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:09.370402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:09.370404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:09.370561 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:09.370966 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:09.370969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:09.370971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:09.371363 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:09.371615 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:09.372004 (+ 389us) server_negotiation.cc:299] Negotiation successful
0926 21:28:09.372196 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 21:28:09.373567 1184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:09.393685 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.393166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58758)
0926 21:28:09.393423 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.393516 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260926 21:28:09.394819 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.394390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58774)
0926 21:28:09.394614 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.394686 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 21:28:09.471477 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1061
I20260926 21:28:09.489077 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 700
I20260926 21:28:09.490754 762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:09.606343 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:09.605842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:38491 (local address 127.0.0.1:58780)
0926 21:28:09.606049 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:09.606164 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:38491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 21:28:09.607003 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 700
2026-09-26T21:28:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5060 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T21:28:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:09Z Disabled control of system clock
I20260926 21:28:09.691936 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40791
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:46203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:40791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 21:28:10.064220 1249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:10.064570 1249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:10.064659 1249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:10.073520 1249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:10.073707 1249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:10.073758 1249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:10.073906 1249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:10.073974 1249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:10.085500 1249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:40791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:10.087533 1249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:10.090301 1249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:10.105058 1255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:10.105247 1256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:10.105221 1258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:10.106807 1249 server_base.cc:1034] running on GCE node
I20260926 21:28:10.107728 1249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:10.109411 1249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:10.110678 1249 hybrid_clock.cc:648] HybridClock initialized: now 1790458090110609 us; error 75 us; skew 500 ppm
I20260926 21:28:10.111093 1249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:10.113760 1249 webserver.cc:460] Webserver started at http://127.14.142.62:46693/ using document root <none> and password file <none>
I20260926 21:28:10.114537 1249 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:10.114681 1249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:10.115023 1249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:10.118261 1249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "98dc15ce3db94bbdbdf589d4903ea887"
format_stamp: "Formatted at 2026-09-26 21:28:10 on dist-test-slave-4d5t"
server_key: "87df8573a18db731a11e940698b5f759"
server_key_iv: "57eee62c04d3c7aa0a0625ffa36f1d6e"
server_key_version: "encryptionkey@0"
I20260926 21:28:10.119160 1249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "98dc15ce3db94bbdbdf589d4903ea887"
format_stamp: "Formatted at 2026-09-26 21:28:10 on dist-test-slave-4d5t"
server_key: "87df8573a18db731a11e940698b5f759"
server_key_iv: "57eee62c04d3c7aa0a0625ffa36f1d6e"
server_key_version: "encryptionkey@0"
I20260926 21:28:10.125082 1249 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 21:28:10.129405 1264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:10.131351 1249 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:28:10.131603 1249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "98dc15ce3db94bbdbdf589d4903ea887"
format_stamp: "Formatted at 2026-09-26 21:28:10 on dist-test-slave-4d5t"
server_key: "87df8573a18db731a11e940698b5f759"
server_key_iv: "57eee62c04d3c7aa0a0625ffa36f1d6e"
server_key_version: "encryptionkey@0"
I20260926 21:28:10.131855 1249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:10.147451 1249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:10.148501 1249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:10.148881 1249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:10.171451 1249 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:40791
I20260926 21:28:10.171474 1315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:40791 every 8 connection(s)
I20260926 21:28:10.173061 1249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:10.177861 1316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:10.181879 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1249
I20260926 21:28:10.182106 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:10.182791 14904 external_mini_cluster.cc:1428] Setting key adf5af598ba79d1b8b34be2cb29fdd73
I20260926 21:28:10.194092 1316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887: Bootstrap starting.
I20260926 21:28:10.198048 1316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:10.199291 1316 log.cc:824] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:10.199692 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:10.185044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.14.142.62:40791)
0926 21:28:10.186593 (+ 1549us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:10.186609 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:10.186793 (+ 184us) server_negotiation.cc:407] Connection header received
0926 21:28:10.188689 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:10.188742 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:10.189331 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:10.189777 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:10.190443 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:10.191822 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:10.193763 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:10.194966 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:10.195257 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:10.196786 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:10.196794 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:10.196810 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:10.197408 (+ 598us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:10.197515 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:10.197538 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:10.197691 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:10.197944 (+ 253us) server_negotiation.cc:299] Negotiation successful
0926 21:28:10.198337 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":554,"threads_started":1}
I20260926 21:28:10.202870 1316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887: No bootstrap required, opened a new log
I20260926 21:28:10.211014 1316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98dc15ce3db94bbdbdf589d4903ea887" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40791 } }
I20260926 21:28:10.211421 1316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:10.211537 1316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98dc15ce3db94bbdbdf589d4903ea887, State: Initialized, Role: FOLLOWER
I20260926 21:28:10.212239 1316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98dc15ce3db94bbdbdf589d4903ea887" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40791 } }
I20260926 21:28:10.212472 1316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:10.212658 1316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:10.212874 1316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:10.215510 1316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98dc15ce3db94bbdbdf589d4903ea887" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40791 } }
I20260926 21:28:10.216090 1316 leader_election.cc:302] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98dc15ce3db94bbdbdf589d4903ea887; no voters:
I20260926 21:28:10.216776 1316 leader_election.cc:288] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:10.216948 1321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:10.217761 1321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [term 1 LEADER]: Becoming Leader. State: Replica: 98dc15ce3db94bbdbdf589d4903ea887, State: Running, Role: LEADER
I20260926 21:28:10.218364 1321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98dc15ce3db94bbdbdf589d4903ea887" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40791 } }
I20260926 21:28:10.219020 1316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:10.221884 1322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98dc15ce3db94bbdbdf589d4903ea887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98dc15ce3db94bbdbdf589d4903ea887" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40791 } } }
I20260926 21:28:10.221891 1323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98dc15ce3db94bbdbdf589d4903ea887. Latest consensus state: current_term: 1 leader_uuid: "98dc15ce3db94bbdbdf589d4903ea887" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98dc15ce3db94bbdbdf589d4903ea887" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 40791 } } }
I20260926 21:28:10.222571 1323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:10.222573 1322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:10.226055 1330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:10.233196 1330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:10.243103 1330 catalog_manager.cc:1347] Generated new cluster ID: 6a3c36663a2248b9a08ef674c9a53fbc
I20260926 21:28:10.243242 1330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:10.255285 1330 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:10.256728 1330 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:10.266117 1330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887: Generated new TSK 0
I20260926 21:28:10.267064 1330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:10.279915 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40791
--builtin_ntp_servers=127.14.142.20:46203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 21:28:10.672744 1340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:10.673058 1340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:10.673234 1340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:10.681334 1340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:10.681452 1340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:10.681627 1340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:28:10.691792 1340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:10.693217 1340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:10.695086 1340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:10.709301 1345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:10.709211 1346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:10.709254 1348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:10.710144 1340 server_base.cc:1034] running on GCE node
I20260926 21:28:10.710777 1340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:10.712002 1340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:10.713467 1340 hybrid_clock.cc:648] HybridClock initialized: now 1790458090713398 us; error 70 us; skew 500 ppm
I20260926 21:28:10.713836 1340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:10.716856 1340 webserver.cc:460] Webserver started at http://127.14.142.1:33991/ using document root <none> and password file <none>
I20260926 21:28:10.717634 1340 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:10.717793 1340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:10.718158 1340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:10.721581 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "987305210cd043c88eb63dcc772d7f34"
format_stamp: "Formatted at 2026-09-26 21:28:10 on dist-test-slave-4d5t"
server_key: "bc1f9f69c673ae2e32c082880a546c5c"
server_key_iv: "cbc4bbd5e4bb5e4b91c3ae51a2e8d186"
server_key_version: "encryptionkey@0"
I20260926 21:28:10.722498 1340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "987305210cd043c88eb63dcc772d7f34"
format_stamp: "Formatted at 2026-09-26 21:28:10 on dist-test-slave-4d5t"
server_key: "bc1f9f69c673ae2e32c082880a546c5c"
server_key_iv: "cbc4bbd5e4bb5e4b91c3ae51a2e8d186"
server_key_version: "encryptionkey@0"
I20260926 21:28:10.729143 1340 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260926 21:28:10.733148 1354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:10.734795 1340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:28:10.735064 1340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "987305210cd043c88eb63dcc772d7f34"
format_stamp: "Formatted at 2026-09-26 21:28:10 on dist-test-slave-4d5t"
server_key: "bc1f9f69c673ae2e32c082880a546c5c"
server_key_iv: "cbc4bbd5e4bb5e4b91c3ae51a2e8d186"
server_key_version: "encryptionkey@0"
I20260926 21:28:10.735343 1340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:10.754753 1340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:10.756070 1340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:10.756493 1340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:10.757984 1340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:10.760313 1340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:10.760524 1340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:10.760748 1340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:10.760864 1340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:10.804335 1340 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:35179
I20260926 21:28:10.804399 1466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:35179 every 8 connection(s)
I20260926 21:28:10.805922 1340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:28:10.814424 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1340
I20260926 21:28:10.814640 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:28:10.815109 14904 external_mini_cluster.cc:1428] Setting key 9635b543ec59840418eaa8a2207e4676
I20260926 21:28:10.818305 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40791
--builtin_ntp_servers=127.14.142.20:46203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 21:28:10.822069 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:10.809461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:51237 (local address 127.14.142.62:40791)
0926 21:28:10.810084 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:10.810089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:10.811780 (+ 1691us) server_negotiation.cc:407] Connection header received
0926 21:28:10.813999 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:10.814007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:10.814120 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:10.814264 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:10.815710 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:10.816728 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:10.818609 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:10.819045 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:10.819198 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:10.821029 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:10.821035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:10.821038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:10.821059 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:10.821103 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:10.821117 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:10.821222 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:10.821510 (+ 288us) server_negotiation.cc:299] Negotiation successful
0926 21:28:10.821629 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":326,"threads_started":1}
I20260926 21:28:10.823371 1470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:10.810257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40791 (local address 127.14.142.1:51237)
0926 21:28:10.811540 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:10.811606 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:10.813644 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:10.814582 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:10.814599 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:10.814749 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:10.815492 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:10.815513 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:10.816932 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:10.816937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:10.818340 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:10.818354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:10.819342 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:10.819353 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:10.819725 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:10.820644 (+ 919us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:10.820648 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:10.820696 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:10.820790 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:10.821237 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:10.821253 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:10.821262 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 21:28:10.821484 (+ 222us) client_negotiation.cc:240] Negotiation successful
0926 21:28:10.821935 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":388,"threads_started":1}
I20260926 21:28:10.825639 1467 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40791
I20260926 21:28:10.826040 1467 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:10.827093 1467 heartbeater.cc:511] Master 127.14.142.62:40791 requested a full tablet report, sending...
I20260926 21:28:10.829924 1281 ts_manager.cc:194] Registered new tserver with Master: 987305210cd043c88eb63dcc772d7f34 (127.14.142.1:35179)
I20260926 21:28:10.832237 1281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.1:51237
W20260926 21:28:11.187248 1471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:11.187680 1471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:11.187783 1471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:11.195427 1471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:11.195559 1471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:11.195673 1471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:28:11.205267 1471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:11.206594 1471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:11.208451 1471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:11.222076 1477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:11.222090 1476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:11.222033 1479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:11.223414 1471 server_base.cc:1034] running on GCE node
I20260926 21:28:11.224094 1471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:11.225378 1471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:11.226660 1471 hybrid_clock.cc:648] HybridClock initialized: now 1790458091226585 us; error 58 us; skew 500 ppm
I20260926 21:28:11.227027 1471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:11.229789 1471 webserver.cc:460] Webserver started at http://127.14.142.2:45387/ using document root <none> and password file <none>
I20260926 21:28:11.230532 1471 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:11.230677 1471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:11.231001 1471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:11.234019 1471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c"
format_stamp: "Formatted at 2026-09-26 21:28:11 on dist-test-slave-4d5t"
server_key: "916840ac675ed8ac85e0b1e8f3151063"
server_key_iv: "f8ffb2f780cd340f9f0c9c1136cf8bd7"
server_key_version: "encryptionkey@0"
I20260926 21:28:11.234855 1471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c"
format_stamp: "Formatted at 2026-09-26 21:28:11 on dist-test-slave-4d5t"
server_key: "916840ac675ed8ac85e0b1e8f3151063"
server_key_iv: "f8ffb2f780cd340f9f0c9c1136cf8bd7"
server_key_version: "encryptionkey@0"
I20260926 21:28:11.241212 1471 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 21:28:11.244931 1485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:11.246592 1471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:28:11.246803 1471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c"
format_stamp: "Formatted at 2026-09-26 21:28:11 on dist-test-slave-4d5t"
server_key: "916840ac675ed8ac85e0b1e8f3151063"
server_key_iv: "f8ffb2f780cd340f9f0c9c1136cf8bd7"
server_key_version: "encryptionkey@0"
I20260926 21:28:11.247045 1471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:11.268673 1471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:11.269550 1471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:11.269866 1471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:11.271082 1471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:11.272782 1471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:11.272904 1471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:11.273039 1471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:11.273133 1471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:11.313383 1471 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:42749
I20260926 21:28:11.313480 1597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:42749 every 8 connection(s)
I20260926 21:28:11.314719 1471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:28:11.320374 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1471
I20260926 21:28:11.320626 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:28:11.321099 14904 external_mini_cluster.cc:1428] Setting key bb426a864d74f286afca9bc2d93f3a49
I20260926 21:28:11.324466 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40791
--builtin_ntp_servers=127.14.142.20:46203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 21:28:11.329769 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.318239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:47123 (local address 127.14.142.62:40791)
0926 21:28:11.318533 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:11.318538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:11.320131 (+ 1593us) server_negotiation.cc:407] Connection header received
0926 21:28:11.322186 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:11.322191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:11.322259 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:11.322366 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:11.323575 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.324759 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.326390 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.327142 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.327286 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.328795 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:11.328805 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:11.328813 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:11.328840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:11.328884 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:11.328903 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:11.328998 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:11.329219 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 21:28:11.329337 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":126}
I20260926 21:28:11.330651 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.318798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40791 (local address 127.14.142.2:47123)
0926 21:28:11.319933 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:11.319999 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:11.321897 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:11.322626 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:11.322638 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:11.322753 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:11.323405 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.323423 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.324925 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.324930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.326229 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.326241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.327367 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.327373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.327590 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.328363 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:11.328366 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:11.328419 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:11.328514 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:11.329012 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:11.329021 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:11.329027 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 21:28:11.329177 (+ 150us) client_negotiation.cc:240] Negotiation successful
0926 21:28:11.329509 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":336,"threads_started":1}
I20260926 21:28:11.332224 1598 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40791
I20260926 21:28:11.332602 1598 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:11.333587 1598 heartbeater.cc:511] Master 127.14.142.62:40791 requested a full tablet report, sending...
I20260926 21:28:11.335225 1281 ts_manager.cc:194] Registered new tserver with Master: 4fa4a9c6a4644ca6b7c7c921e6e27e0c (127.14.142.2:42749)
I20260926 21:28:11.336169 1281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.2:47123
W20260926 21:28:11.678220 1601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:11.678614 1601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:11.678767 1601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:11.687222 1601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:11.687422 1601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:11.687852 1601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:28:11.697697 1601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:40791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:11.699091 1601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:11.700857 1601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:11.715106 1607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:11.715260 1609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:11.715237 1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:11.716812 1601 server_base.cc:1034] running on GCE node
I20260926 21:28:11.717525 1601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:11.718727 1601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:11.720007 1601 hybrid_clock.cc:648] HybridClock initialized: now 1790458091719913 us; error 84 us; skew 500 ppm
I20260926 21:28:11.720481 1601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:11.729797 1601 webserver.cc:460] Webserver started at http://127.14.142.3:36577/ using document root <none> and password file <none>
I20260926 21:28:11.730602 1601 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:11.730710 1601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:11.730989 1601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:11.734283 1601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "0856cb067bcb4793ac43fb5077c5e017"
format_stamp: "Formatted at 2026-09-26 21:28:11 on dist-test-slave-4d5t"
server_key: "d90e1d03f745de2ece3132dec0662b3f"
server_key_iv: "c419b8dbd0346a9b5d5f5f4080dcba2a"
server_key_version: "encryptionkey@0"
I20260926 21:28:11.735184 1601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "0856cb067bcb4793ac43fb5077c5e017"
format_stamp: "Formatted at 2026-09-26 21:28:11 on dist-test-slave-4d5t"
server_key: "d90e1d03f745de2ece3132dec0662b3f"
server_key_iv: "c419b8dbd0346a9b5d5f5f4080dcba2a"
server_key_version: "encryptionkey@0"
I20260926 21:28:11.741425 1601 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260926 21:28:11.746235 1615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:11.748217 1601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:28:11.748525 1601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "0856cb067bcb4793ac43fb5077c5e017"
format_stamp: "Formatted at 2026-09-26 21:28:11 on dist-test-slave-4d5t"
server_key: "d90e1d03f745de2ece3132dec0662b3f"
server_key_iv: "c419b8dbd0346a9b5d5f5f4080dcba2a"
server_key_version: "encryptionkey@0"
I20260926 21:28:11.748947 1601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:11.769958 1601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:11.770860 1601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:11.771189 1601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:11.772248 1601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:11.773856 1601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:11.773981 1601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:11.774101 1601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:11.774196 1601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:11.813205 1601 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:34587
I20260926 21:28:11.813241 1727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:34587 every 8 connection(s)
I20260926 21:28:11.814555 1601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:28:11.825183 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1601
I20260926 21:28:11.825511 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:28:11.826160 14904 external_mini_cluster.cc:1428] Setting key f3243729dd6ff404e41b18f4ea4c0115
I20260926 21:28:11.830641 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.817342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:53105 (local address 127.14.142.62:40791)
0926 21:28:11.817695 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:11.817702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:11.819387 (+ 1685us) server_negotiation.cc:407] Connection header received
0926 21:28:11.821882 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:11.821887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:11.821997 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:11.822123 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:11.824009 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.824886 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.827486 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.828005 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.828180 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.829754 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:11.829759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:11.829762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:11.829783 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:11.829814 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:11.829825 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:11.829926 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:11.830156 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 21:28:11.830262 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260926 21:28:11.831696 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.817822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:40791 (local address 127.14.142.3:53105)
0926 21:28:11.819127 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:11.819179 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:11.821409 (+ 2230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:11.822436 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:11.822446 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:11.822575 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:11.823736 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.823760 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.825204 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.825224 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.827254 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.827278 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.828278 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.828285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.828603 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.829372 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:11.829375 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:11.829419 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:11.829524 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:11.829925 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:11.829936 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:11.829947 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 21:28:11.830148 (+ 201us) client_negotiation.cc:240] Negotiation successful
0926 21:28:11.830595 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":378,"threads_started":1}
I20260926 21:28:11.833350 1728 heartbeater.cc:347] Connected to a master server at 127.14.142.62:40791
I20260926 21:28:11.833724 1728 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:11.834597 1728 heartbeater.cc:511] Master 127.14.142.62:40791 requested a full tablet report, sending...
I20260926 21:28:11.836530 1467 heartbeater.cc:503] Master 127.14.142.62:40791 was elected leader, sending a full tablet report...
I20260926 21:28:11.837237 1281 ts_manager.cc:194] Registered new tserver with Master: 0856cb067bcb4793ac43fb5077c5e017 (127.14.142.3:34587)
I20260926 21:28:11.838210 1281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.3:53105
I20260926 21:28:11.842670 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:28:11.854622 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.847363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.14.142.62:40791)
0926 21:28:11.847686 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:11.847693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:11.847992 (+ 299us) server_negotiation.cc:407] Connection header received
0926 21:28:11.848144 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:11.848148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:11.848215 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:11.848300 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:11.848981 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.849802 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.851445 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.851986 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.852099 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.853734 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:11.853740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:11.853744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:11.853765 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:11.853799 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:11.853812 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:11.853941 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:11.854187 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 21:28:11.854259 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 21:28:11.859556 1281 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:28:11.868208 1281 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:28:11.897208 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.882790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44516)
0926 21:28:11.885150 (+ 2360us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:11.885203 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:11.885430 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:11.889439 (+ 4009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:11.889445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:11.889471 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:11.890030 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.890047 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.892325 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.892330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.894132 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.894145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.896166 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.896172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.896374 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.896435 (+ 61us) client_negotiation.cc:769] Sending connection context
0926 21:28:11.896618 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 21:28:11.896820 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1967,"thread_start_us":1791,"threads_started":1}
I20260926 21:28:11.897250 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.883491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44516 (local address 127.14.142.1:35179)
0926 21:28:11.888615 (+ 5124us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:11.888628 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:11.888663 (+ 35us) server_negotiation.cc:407] Connection header received
0926 21:28:11.888795 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:11.888807 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:11.889171 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:11.890263 (+ 1092us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:11.890508 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.892170 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.894335 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.895948 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.896112 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.896418 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:11.896669 (+ 251us) server_negotiation.cc:299] Negotiation successful
0926 21:28:11.896855 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4869,"thread_start_us":4693,"threads_started":1}
I20260926 21:28:11.897909 1742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.882880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:42749 (local address 127.0.0.1:51854)
0926 21:28:11.887936 (+ 5056us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:11.887961 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:11.888964 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:11.889785 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:11.889793 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:11.889818 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:11.890320 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.890336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.891992 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.891997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.894535 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.894551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.896594 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.896601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.896790 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.896826 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 21:28:11.896916 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 21:28:11.897055 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4876,"thread_start_us":6295,"threads_started":1}
I20260926 21:28:11.898288 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.883493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.14.142.2:42749)
0926 21:28:11.884364 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:11.884378 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:11.888103 (+ 3725us) server_negotiation.cc:407] Connection header received
0926 21:28:11.889205 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:11.889214 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:11.889478 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:11.889649 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:11.890525 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.891798 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.894752 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.896154 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.896274 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.896470 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:11.897759 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0926 21:28:11.897941 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":551,"threads_started":1}
I20260926 21:28:11.898548 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.883783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.14.142.3:34587)
0926 21:28:11.888987 (+ 5204us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:11.888996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:11.889578 (+ 582us) server_negotiation.cc:407] Connection header received
0926 21:28:11.889774 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:11.889782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:11.889995 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:11.890192 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:11.891180 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.892718 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.895520 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.896871 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.897002 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.897249 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:11.898002 (+ 753us) server_negotiation.cc:299] Negotiation successful
0926 21:28:11.898162 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5035,"thread_start_us":4863,"threads_started":1}
I20260926 21:28:11.898737 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:11.883177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:34587 (local address 127.0.0.1:41012)
0926 21:28:11.889437 (+ 6260us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:11.889456 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:11.889592 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:11.890471 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:11.890476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:11.890500 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:11.890953 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.890967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.892865 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.892874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.895277 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:11.895289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:11.897380 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:11.897387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:11.897598 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:11.897631 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 21:28:11.897713 (+ 82us) client_negotiation.cc:240] Negotiation successful
0926 21:28:11.897837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6133,"thread_start_us":4388,"threads_started":1}
I20260926 21:28:11.901537 1533 tablet_service.cc:1467] Processing CreateTablet for tablet df20940228ea4975bec92124b3625aa8 (DEFAULT_TABLE table=test-table [id=2b967d10e8dc452f9039067bf0d0ea97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:11.901551 1402 tablet_service.cc:1467] Processing CreateTablet for tablet df20940228ea4975bec92124b3625aa8 (DEFAULT_TABLE table=test-table [id=2b967d10e8dc452f9039067bf0d0ea97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:11.901935 1663 tablet_service.cc:1467] Processing CreateTablet for tablet df20940228ea4975bec92124b3625aa8 (DEFAULT_TABLE table=test-table [id=2b967d10e8dc452f9039067bf0d0ea97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:11.902880 1533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df20940228ea4975bec92124b3625aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:11.902966 1402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df20940228ea4975bec92124b3625aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:11.903245 1663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df20940228ea4975bec92124b3625aa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:11.915556 1746 tablet_bootstrap.cc:492] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Bootstrap starting.
I20260926 21:28:11.917799 1747 tablet_bootstrap.cc:492] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Bootstrap starting.
I20260926 21:28:11.919838 1746 tablet_bootstrap.cc:654] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:11.921331 1748 tablet_bootstrap.cc:492] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Bootstrap starting.
I20260926 21:28:11.921653 1746 log.cc:824] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:11.922220 1747 tablet_bootstrap.cc:654] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:11.924135 1747 log.cc:824] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:11.925312 1748 tablet_bootstrap.cc:654] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:11.926304 1748 log.cc:824] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:11.926847 1746 tablet_bootstrap.cc:492] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: No bootstrap required, opened a new log
I20260926 21:28:11.927181 1746 ts_tablet_manager.cc:1397] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260926 21:28:11.927374 1747 tablet_bootstrap.cc:492] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: No bootstrap required, opened a new log
I20260926 21:28:11.927758 1747 ts_tablet_manager.cc:1397] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260926 21:28:11.929082 1748 tablet_bootstrap.cc:492] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: No bootstrap required, opened a new log
I20260926 21:28:11.929332 1748 ts_tablet_manager.cc:1397] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 21:28:11.937433 1748 raft_consensus.cc:348] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:11.937853 1748 raft_consensus.cc:374] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:11.937954 1748 raft_consensus.cc:729] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0856cb067bcb4793ac43fb5077c5e017, State: Initialized, Role: FOLLOWER
I20260926 21:28:11.938633 1748 consensus_queue.cc:260] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:11.939795 1728 heartbeater.cc:503] Master 127.14.142.62:40791 was elected leader, sending a full tablet report...
I20260926 21:28:11.939678 1746 raft_consensus.cc:348] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:11.939710 1747 raft_consensus.cc:348] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:11.940178 1747 raft_consensus.cc:374] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:11.940178 1746 raft_consensus.cc:374] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:11.940291 1747 raft_consensus.cc:729] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 987305210cd043c88eb63dcc772d7f34, State: Initialized, Role: FOLLOWER
I20260926 21:28:11.940296 1746 raft_consensus.cc:729] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa4a9c6a4644ca6b7c7c921e6e27e0c, State: Initialized, Role: FOLLOWER
I20260926 21:28:11.941030 1746 consensus_queue.cc:260] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:11.941149 1747 consensus_queue.cc:260] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:11.942034 1748 ts_tablet_manager.cc:1428] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260926 21:28:11.948338 1598 heartbeater.cc:503] Master 127.14.142.62:40791 was elected leader, sending a full tablet report...
I20260926 21:28:11.949268 1746 ts_tablet_manager.cc:1428] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Time spent starting tablet: real 0.022s user 0.019s sys 0.005s
I20260926 21:28:11.949823 1747 ts_tablet_manager.cc:1428] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Time spent starting tablet: real 0.022s user 0.015s sys 0.007s
I20260926 21:28:12.024505 1752 raft_consensus.cc:482] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:12.025012 1752 raft_consensus.cc:504] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:12.027271 1752 leader_election.cc:288] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fa4a9c6a4644ca6b7c7c921e6e27e0c (127.14.142.2:42749), 987305210cd043c88eb63dcc772d7f34 (127.14.142.1:35179)
I20260926 21:28:12.034514 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.027459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:42749 (local address 127.14.142.3:58301)
0926 21:28:12.027724 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.027750 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:12.027935 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:12.028418 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:12.028423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:12.028451 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:12.028852 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:12.028869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.030599 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.030603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:12.032631 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:12.032649 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.033897 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.033902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:12.034040 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.034075 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 21:28:12.034151 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 21:28:12.034263 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260926 21:28:12.034772 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.027769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:58301 (local address 127.14.142.2:42749)
0926 21:28:12.028060 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:12.028070 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:12.028103 (+ 33us) server_negotiation.cc:407] Connection header received
0926 21:28:12.028172 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:12.028176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:12.028250 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:12.028369 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:12.029153 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.030419 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.032850 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.033732 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.033905 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.033981 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:12.034208 (+ 227us) server_negotiation.cc:299] Negotiation successful
0926 21:28:12.034344 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 21:28:12.035915 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.028009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.14.142.3:48771)
0926 21:28:12.029203 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.029219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:12.029321 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:12.029698 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:12.029701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:12.029722 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:12.030018 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:12.030028 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.031567 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.031573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:12.033472 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:12.033484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.035297 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.035307 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:12.035483 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.035509 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 21:28:12.035595 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 21:28:12.035690 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"thread_start_us":776,"threads_started":1}
I20260926 21:28:12.036037 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.028292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:48771 (local address 127.14.142.1:35179)
0926 21:28:12.028640 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:12.028651 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:12.029324 (+ 673us) server_negotiation.cc:407] Connection header received
0926 21:28:12.029426 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:12.029431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:12.029545 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:12.029676 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:12.030331 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.031367 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.033716 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.035104 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.035335 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.035410 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:12.035655 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 21:28:12.035798 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260926 21:28:12.035945 1553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df20940228ea4975bec92124b3625aa8" candidate_uuid: "0856cb067bcb4793ac43fb5077c5e017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" is_pre_election: true
I20260926 21:28:12.036671 1553 raft_consensus.cc:2454] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0856cb067bcb4793ac43fb5077c5e017 in term 0.
I20260926 21:28:12.037194 1422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df20940228ea4975bec92124b3625aa8" candidate_uuid: "0856cb067bcb4793ac43fb5077c5e017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "987305210cd043c88eb63dcc772d7f34" is_pre_election: true
I20260926 21:28:12.037528 1618 leader_election.cc:302] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0856cb067bcb4793ac43fb5077c5e017, 4fa4a9c6a4644ca6b7c7c921e6e27e0c; no voters:
I20260926 21:28:12.037720 1422 raft_consensus.cc:2454] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0856cb067bcb4793ac43fb5077c5e017 in term 0.
I20260926 21:28:12.038038 1752 raft_consensus.cc:2781] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:28:12.038254 1752 raft_consensus.cc:482] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:28:12.038398 1752 raft_consensus.cc:3037] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:12.040977 1752 raft_consensus.cc:504] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:12.041697 1752 leader_election.cc:288] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [CANDIDATE]: Term 1 election: Requested vote from peers 4fa4a9c6a4644ca6b7c7c921e6e27e0c (127.14.142.2:42749), 987305210cd043c88eb63dcc772d7f34 (127.14.142.1:35179)
I20260926 21:28:12.042162 1553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df20940228ea4975bec92124b3625aa8" candidate_uuid: "0856cb067bcb4793ac43fb5077c5e017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c"
I20260926 21:28:12.042258 1422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df20940228ea4975bec92124b3625aa8" candidate_uuid: "0856cb067bcb4793ac43fb5077c5e017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "987305210cd043c88eb63dcc772d7f34"
I20260926 21:28:12.042407 1553 raft_consensus.cc:3037] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:12.042450 1422 raft_consensus.cc:3037] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:12.045841 1422 raft_consensus.cc:2454] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0856cb067bcb4793ac43fb5077c5e017 in term 1.
I20260926 21:28:12.045841 1553 raft_consensus.cc:2454] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0856cb067bcb4793ac43fb5077c5e017 in term 1.
I20260926 21:28:12.046382 1616 leader_election.cc:302] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0856cb067bcb4793ac43fb5077c5e017, 987305210cd043c88eb63dcc772d7f34; no voters:
I20260926 21:28:12.046921 1752 raft_consensus.cc:2781] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:12.047528 1752 raft_consensus.cc:686] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 1 LEADER]: Becoming Leader. State: Replica: 0856cb067bcb4793ac43fb5077c5e017, State: Running, Role: LEADER
I20260926 21:28:12.048103 1752 consensus_queue.cc:237] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } }
I20260926 21:28:12.054330 1280 catalog_manager.cc:5495] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0856cb067bcb4793ac43fb5077c5e017 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "0856cb067bcb4793ac43fb5077c5e017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0856cb067bcb4793ac43fb5077c5e017" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 34587 } health_report { overall_health: HEALTHY } } }
W20260926 21:28:12.059137 1468 tablet.cc:2369] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:28:12.066327 1729 tablet.cc:2369] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 21:28:12.067132 1599 tablet.cc:2369] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:12.091580 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.085621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.14.142.3:34587)
0926 21:28:12.085790 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:12.085794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:12.085888 (+ 94us) server_negotiation.cc:407] Connection header received
0926 21:28:12.086063 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:12.086067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:12.086157 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:12.086295 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:28:12.086675 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.087404 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.089112 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.089588 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.089714 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.090423 (+ 709us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:28:12.090945 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:28:12.091090 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:12.091335 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 21:28:12.091375 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 21:28:12.100001 1422 raft_consensus.cc:1261] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 1 FOLLOWER]: Refusing update from remote peer 0856cb067bcb4793ac43fb5077c5e017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:12.100001 1553 raft_consensus.cc:1261] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 1 FOLLOWER]: Refusing update from remote peer 0856cb067bcb4793ac43fb5077c5e017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:12.101037 1756 consensus_queue.cc:1035] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fa4a9c6a4644ca6b7c7c921e6e27e0c" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 42749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:12.101485 1752 consensus_queue.cc:1035] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "987305210cd043c88eb63dcc772d7f34" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 35179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:12.113598 1759 mvcc.cc:205] Tried to move back new op lower bound from 7333716345231446016 to 7333716345033728000. Current Snapshot: MvccSnapshot[applied={T|T < 7333716345231446016}]
I20260926 21:28:12.133159 1280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:12.133481 1280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:12.139256 1280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98dc15ce3db94bbdbdf589d4903ea887: Sending DeleteTablet for 3 replicas of tablet df20940228ea4975bec92124b3625aa8
I20260926 21:28:12.140390 1533 tablet_service.cc:1514] Processing DeleteTablet for tablet df20940228ea4975bec92124b3625aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:12 UTC) from {username='slave'} at 127.0.0.1:51854
I20260926 21:28:12.140403 1663 tablet_service.cc:1514] Processing DeleteTablet for tablet df20940228ea4975bec92124b3625aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:12 UTC) from {username='slave'} at 127.0.0.1:41012
I20260926 21:28:12.141436 1769 tablet_replica.cc:330] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: stopping tablet replica
I20260926 21:28:12.142151 1769 raft_consensus.cc:2229] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:28:12.142426 1402 tablet_service.cc:1514] Processing DeleteTablet for tablet df20940228ea4975bec92124b3625aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:12 UTC) from {username='slave'} at 127.0.0.1:44516
I20260926 21:28:12.142539 1663 tablet_service.cc:1514] Processing DeleteTablet for tablet df20940228ea4975bec92124b3625aa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:12 UTC) from {username='slave'} at 127.0.0.1:41012
I20260926 21:28:12.142864 1769 raft_consensus.cc:2258] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:12.146400 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1340
I20260926 21:28:12.153457 1769 ts_tablet_manager.cc:1905] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:12.160815 1770 tablet_replica.cc:330] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: stopping tablet replica
I20260926 21:28:12.161541 1770 raft_consensus.cc:2229] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:12.162213 1770 raft_consensus.cc:2258] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:12.164721 1770 ts_tablet_manager.cc:1905] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:12.172127 1770 ts_tablet_manager.cc:1918] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:12.172788 1771 tablet_replica.cc:330] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: stopping tablet replica
I20260926 21:28:12.173247 1771 raft_consensus.cc:2229] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:12.173797 1771 raft_consensus.cc:2258] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:12.175851 1769 ts_tablet_manager.cc:1918] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:12.175999 1771 ts_tablet_manager.cc:1905] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:12.176065 1769 log.cc:1196] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/df20940228ea4975bec92124b3625aa8
I20260926 21:28:12.176661 1769 ts_tablet_manager.cc:1939] T df20940228ea4975bec92124b3625aa8 P 0856cb067bcb4793ac43fb5077c5e017: Deleting consensus metadata
I20260926 21:28:12.178608 1268 catalog_manager.cc:4841] TS 0856cb067bcb4793ac43fb5077c5e017 (127.14.142.3:34587): tablet df20940228ea4975bec92124b3625aa8 (table test-table [id=2b967d10e8dc452f9039067bf0d0ea97]) successfully deleted
I20260926 21:28:12.181273 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.147798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44530 (local address 127.14.142.1:35179)
0926 21:28:12.148046 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:12.148055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:12.167943 (+ 19888us) server_negotiation.cc:407] Connection header received
0926 21:28:12.168599 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:12.168609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:12.168767 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:12.168871 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:12.169819 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.171380 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.173349 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.174070 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.174198 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.179346 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:12.179359 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:12.179378 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:12.180173 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:12.180245 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:12.180277 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:12.180393 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:12.180707 (+ 314us) server_negotiation.cc:299] Negotiation successful
0926 21:28:12.180837 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:28:12.182444 1462 generic_service.cc:196] Checking for leaks (request via RPC)
W20260926 21:28:12.183269 1268 catalog_manager.cc:4808] TS 0856cb067bcb4793ac43fb5077c5e017 (127.14.142.3:34587): delete failed for tablet df20940228ea4975bec92124b3625aa8 because the tablet was not found. No further retry: Not found: Tablet not found: df20940228ea4975bec92124b3625aa8
I20260926 21:28:12.183806 1770 log.cc:1196] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/df20940228ea4975bec92124b3625aa8
I20260926 21:28:12.184451 1770 ts_tablet_manager.cc:1939] T df20940228ea4975bec92124b3625aa8 P 4fa4a9c6a4644ca6b7c7c921e6e27e0c: Deleting consensus metadata
I20260926 21:28:12.186474 1267 catalog_manager.cc:4841] TS 4fa4a9c6a4644ca6b7c7c921e6e27e0c (127.14.142.2:42749): tablet df20940228ea4975bec92124b3625aa8 (table test-table [id=2b967d10e8dc452f9039067bf0d0ea97]) successfully deleted
I20260926 21:28:12.307732 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1340
I20260926 21:28:12.307732 1771 ts_tablet_manager.cc:1918] T df20940228ea4975bec92124b3625aa8 P 987305210cd043c88eb63dcc772d7f34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:12.332047 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1471
W20260926 21:28:12.332007 1265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.142.1:35179 (error 108)
I20260926 21:28:12.346638 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.334159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.14.142.2:42749)
0926 21:28:12.334415 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:12.334425 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:12.334451 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:28:12.334537 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:12.334546 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:12.334727 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:12.334868 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:12.335828 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.338078 (+ 2250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.341498 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.342515 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.342688 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.344509 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:12.344518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:12.344530 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:12.345267 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:12.345317 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:12.345337 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:12.345490 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:12.346108 (+ 618us) server_negotiation.cc:299] Negotiation successful
0926 21:28:12.346235 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 21:28:12.347033 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.346384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44544)
0926 21:28:12.346708 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.346848 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
W20260926 21:28:12.347329 1265 catalog_manager.cc:4584] TS 987305210cd043c88eb63dcc772d7f34 (127.14.142.1:35179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df20940228ea4975bec92124b3625aa8: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
I20260926 21:28:12.347877 1593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:12.400986 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.400402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44560)
0926 21:28:12.400694 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.400808 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260926 21:28:12.402197 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.401788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44564)
0926 21:28:12.401957 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.402052 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 21:28:12.441250 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1471
I20260926 21:28:12.460018 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1601
I20260926 21:28:12.463747 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.463041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44576)
0926 21:28:12.463335 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.463473 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260926 21:28:12.465540 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.464791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44590)
0926 21:28:12.465024 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.465278 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260926 21:28:12.469810 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.460954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.14.142.3:34587)
0926 21:28:12.461173 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:12.461180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:12.461202 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:28:12.461422 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:12.461427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:12.461538 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:12.461659 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:12.462060 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.463261 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.465865 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:12.466484 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:12.466604 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:12.468026 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:12.468036 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:12.468051 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:12.468826 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:12.468883 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:12.468914 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:12.469022 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:12.469257 (+ 235us) server_negotiation.cc:299] Negotiation successful
0926 21:28:12.469356 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260926 21:28:12.470659 1723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:12.547160 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.546398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44596)
0926 21:28:12.546806 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.546948 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260926 21:28:12.548595 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:12.548008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:35179 (local address 127.0.0.1:44598)
0926 21:28:12.548270 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:12.548409 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:35179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260926 21:28:12.597446 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1601
I20260926 21:28:12.618870 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1249
I20260926 21:28:12.620000 1311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:12.735096 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1249
2026-09-26T21:28:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3127 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:12 dist-test-slave-4d5t krb5kdc[1780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:12 dist-test-slave-4d5t krb5kdc[1780](info): set up 2 sockets
Sep 26 21:28:12 dist-test-slave-4d5t krb5kdc[1780](info): commencing operation
krb5kdc: starting...
W20260926 21:28:13.885466 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.004s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:14 dist-test-slave-4d5t krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:14.185905 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:35455
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:35455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:14.539983 1793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:14.540313 1793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:14.540493 1793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:14.548310 1793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:14.548444 1793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:14.548539 1793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:14.548658 1793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:14.548767 1793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:14.559087 1793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:35455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:35455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:14.560483 1793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:14.561834 1793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:14.574891 1798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:14.574857 1799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:14.575039 1801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:14.575903 1793 server_base.cc:1034] running on GCE node
I20260926 21:28:14.576756 1793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:14.578249 1793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:14.579465 1793 hybrid_clock.cc:648] HybridClock initialized: now 1790458094579420 us; error 33 us; skew 500 ppm
I20260926 21:28:14.579804 1793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:14 dist-test-slave-4d5t krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458094, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:14.583004 1793 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:14.584746 1793 webserver.cc:460] Webserver started at http://127.14.142.62:41587/ using document root <none> and password file <none>
I20260926 21:28:14.585443 1793 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:14.585546 1793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:14.585805 1793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:14.588955 1793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "02cc0403f86e47b8ac341694f25b48f7"
format_stamp: "Formatted at 2026-09-26 21:28:14 on dist-test-slave-4d5t"
server_key: "6564e2f5c0b73dba222a5a3075056b00"
server_key_iv: "ed48f420a3c98629c1a8a5d3a2f4d613"
server_key_version: "encryptionkey@0"
I20260926 21:28:14.589694 1793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "02cc0403f86e47b8ac341694f25b48f7"
format_stamp: "Formatted at 2026-09-26 21:28:14 on dist-test-slave-4d5t"
server_key: "6564e2f5c0b73dba222a5a3075056b00"
server_key_iv: "ed48f420a3c98629c1a8a5d3a2f4d613"
server_key_version: "encryptionkey@0"
I20260926 21:28:14.595368 1793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:28:14.599193 1808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:14.600905 1793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 21:28:14.601092 1793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "02cc0403f86e47b8ac341694f25b48f7"
format_stamp: "Formatted at 2026-09-26 21:28:14 on dist-test-slave-4d5t"
server_key: "6564e2f5c0b73dba222a5a3075056b00"
server_key_iv: "ed48f420a3c98629c1a8a5d3a2f4d613"
server_key_version: "encryptionkey@0"
I20260926 21:28:14.601291 1793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:14.621153 1793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:14.626235 1793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:14.626538 1793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:14.649395 1793 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:35455
I20260926 21:28:14.649398 1859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:35455 every 8 connection(s)
I20260926 21:28:14.651003 1793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:14.654754 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1793
I20260926 21:28:14.655006 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:14.655206 1860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:14.656054 14904 external_mini_cluster.cc:1428] Setting key 4f4ec8dfea9d17900800701a5f2f412a
I20260926 21:28:14.670244 1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7: Bootstrap starting.
Sep 26 21:28:14 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:14.673985 1860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:14.675299 1860 log.cc:824] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:14.678706 1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7: No bootstrap required, opened a new log
I20260926 21:28:14.682785 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:14.658559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.14.142.62:35455)
0926 21:28:14.659471 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:14.659484 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:14.659527 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:28:14.662576 (+ 3049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:14.662602 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:14.663099 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:14.663618 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:14.665404 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:14.666848 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:14.668952 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:14.669717 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:14.669958 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:14.674516 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:14.674526 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:14.674542 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:14.674584 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:14.678157 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:14.679191 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:14.679198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:14.679207 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:14.679382 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:14.679822 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:14.679826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:14.679828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:14.680391 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:14.680767 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:14.681267 (+ 500us) server_negotiation.cc:299] Negotiation successful
0926 21:28:14.681715 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":408,"threads_started":1}
I20260926 21:28:14.688162 1860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02cc0403f86e47b8ac341694f25b48f7" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35455 } }
I20260926 21:28:14.688625 1860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:14.688746 1860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02cc0403f86e47b8ac341694f25b48f7, State: Initialized, Role: FOLLOWER
I20260926 21:28:14.689412 1860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02cc0403f86e47b8ac341694f25b48f7" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35455 } }
I20260926 21:28:14.689682 1860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:14.689850 1860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:14.690038 1860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:14.692507 1860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02cc0403f86e47b8ac341694f25b48f7" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35455 } }
I20260926 21:28:14.693143 1860 leader_election.cc:302] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02cc0403f86e47b8ac341694f25b48f7; no voters:
I20260926 21:28:14.693962 1860 leader_election.cc:288] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:14.694281 1865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:14.695395 1865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [term 1 LEADER]: Becoming Leader. State: Replica: 02cc0403f86e47b8ac341694f25b48f7, State: Running, Role: LEADER
I20260926 21:28:14.696147 1865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02cc0403f86e47b8ac341694f25b48f7" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35455 } }
I20260926 21:28:14.696697 1860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:14.699427 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02cc0403f86e47b8ac341694f25b48f7. Latest consensus state: current_term: 1 leader_uuid: "02cc0403f86e47b8ac341694f25b48f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02cc0403f86e47b8ac341694f25b48f7" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35455 } } }
I20260926 21:28:14.700078 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:14.699986 1866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02cc0403f86e47b8ac341694f25b48f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02cc0403f86e47b8ac341694f25b48f7" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 35455 } } }
I20260926 21:28:14.701401 1866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:14.710440 1874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:14.715641 1874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:14.724452 1874 catalog_manager.cc:1347] Generated new cluster ID: f21bf727ab4c4475bec7778a5e46dd05
I20260926 21:28:14.724601 1874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:14.740018 1874 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:14.742283 1874 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:14.758805 1874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7: Generated new TSK 0
I20260926 21:28:14.759898 1874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.14.142.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
Entry for principal kudu/127.14.142.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab.
I20260926 21:28:14.836680 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:35455
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:15.216199 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:15.216565 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:15.216780 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:15.224426 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:15.224627 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:15.224804 1886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:28:15.236259 1886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.1.keytab
--principal=kudu/127.14.142.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:35455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:15.238178 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:15.239598 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:15.253391 1894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:15.253481 1892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:15.253458 1891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:15.254398 1886 server_base.cc:1034] running on GCE node
I20260926 21:28:15.255059 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:15.256104 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:15.257359 1886 hybrid_clock.cc:648] HybridClock initialized: now 1790458095257294 us; error 62 us; skew 500 ppm
I20260926 21:28:15.257742 1886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:15 dist-test-slave-4d5t krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458095, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:15.261554 1886 init.cc:377] Logged in from keytab as kudu/127.14.142.1@KRBTEST.COM (short username kudu)
I20260926 21:28:15.263515 1886 webserver.cc:460] Webserver started at http://127.14.142.1:43155/ using document root <none> and password file <none>
I20260926 21:28:15.264333 1886 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:15.264503 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:15.264878 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:15.267896 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "89a47a31839e4a8fb02b78a2a10a6570"
format_stamp: "Formatted at 2026-09-26 21:28:15 on dist-test-slave-4d5t"
server_key: "d80228d9440bbcb6ea8da86b562ce7eb"
server_key_iv: "621851de171a424488db25765f849d32"
server_key_version: "encryptionkey@0"
I20260926 21:28:15.268779 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "89a47a31839e4a8fb02b78a2a10a6570"
format_stamp: "Formatted at 2026-09-26 21:28:15 on dist-test-slave-4d5t"
server_key: "d80228d9440bbcb6ea8da86b562ce7eb"
server_key_iv: "621851de171a424488db25765f849d32"
server_key_version: "encryptionkey@0"
I20260926 21:28:15.275213 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:28:15.279089 1901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:15.280844 1886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 21:28:15.281045 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "89a47a31839e4a8fb02b78a2a10a6570"
format_stamp: "Formatted at 2026-09-26 21:28:15 on dist-test-slave-4d5t"
server_key: "d80228d9440bbcb6ea8da86b562ce7eb"
server_key_iv: "621851de171a424488db25765f849d32"
server_key_version: "encryptionkey@0"
I20260926 21:28:15.281275 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:15.300983 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:15.305608 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:15.305907 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:15.307029 1886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:15.308423 1886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:15.308542 1886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:15.308704 1886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:15.308780 1886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:15.347136 1886 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:42099
I20260926 21:28:15.347190 2013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:42099 every 8 connection(s)
I20260926 21:28:15.348435 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:28:15.353205 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 1886
I20260926 21:28:15.353485 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:28:15.354125 14904 external_mini_cluster.cc:1428] Setting key f22802f36e21969cc0a782417c06cdc1
Sep 26 21:28:15 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458095, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.1@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:15.372862 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:15.351253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:39351 (local address 127.14.142.62:35455)
0926 21:28:15.351969 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:15.351977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:15.352847 (+ 870us) server_negotiation.cc:407] Connection header received
0926 21:28:15.355707 (+ 2860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:15.355714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:15.355815 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:15.355970 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:15.358196 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.358987 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.361564 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.361988 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.362104 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:15.366549 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:15.366555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:15.366558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:15.366592 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:15.369656 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:15.370423 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:15.370430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:15.370443 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:15.370558 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:15.371012 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:15.371018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:15.371021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:15.371313 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:15.371474 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:15.372217 (+ 743us) server_negotiation.cc:299] Negotiation successful
0926 21:28:15.372461 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":349,"threads_started":1}
I20260926 21:28:15.373869 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:15.351389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:35455 (local address 127.14.142.1:39351)
0926 21:28:15.352493 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:15.352655 (+ 162us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:15.355342 (+ 2687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:15.356229 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:15.356237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:15.357077 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:15.357952 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:15.357980 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.359170 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.359178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:15.361371 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:15.361391 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.362239 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.362248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:15.362597 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:15.363422 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:15.363464 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:15.366278 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:15.369815 (+ 3537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:15.369824 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:15.369843 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:15.370283 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:15.370675 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:15.370680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:15.370684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:15.370874 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:15.371441 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:15.371449 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:15.371828 (+ 379us) client_negotiation.cc:769] Sending connection context
0926 21:28:15.372109 (+ 281us) client_negotiation.cc:240] Negotiation successful
0926 21:28:15.372690 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":410,"threads_started":1}
I20260926 21:28:15.375887 2014 heartbeater.cc:347] Connected to a master server at 127.14.142.62:35455
I20260926 21:28:15.376343 2014 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:15.377506 2014 heartbeater.cc:511] Master 127.14.142.62:35455 requested a full tablet report, sending...
I20260926 21:28:15.379873 1825 ts_manager.cc:194] Registered new tserver with Master: 89a47a31839e4a8fb02b78a2a10a6570 (127.14.142.1:42099)
I20260926 21:28:15.381441 1825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.1@KRBTEST.COM'} at 127.14.142.1:39351
WARNING: no policy specified for kudu/127.14.142.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
Entry for principal kudu/127.14.142.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab.
I20260926 21:28:15.416662 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:35455
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:15.770772 2020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:15.771180 2020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:15.771390 2020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:15.780714 2020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:15.780869 2020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:15.781073 2020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:28:15.792886 2020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.2.keytab
--principal=kudu/127.14.142.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:35455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:15.794252 2020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:15.795547 2020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:15.809878 2025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:15.809903 2028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:15.809809 2026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:15.811045 2020 server_base.cc:1034] running on GCE node
I20260926 21:28:15.811818 2020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:15.812997 2020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:15.814265 2020 hybrid_clock.cc:648] HybridClock initialized: now 1790458095814208 us; error 53 us; skew 500 ppm
I20260926 21:28:15.814652 2020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:15 dist-test-slave-4d5t krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458095, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:15.818694 2020 init.cc:377] Logged in from keytab as kudu/127.14.142.2@KRBTEST.COM (short username kudu)
I20260926 21:28:15.821137 2020 webserver.cc:460] Webserver started at http://127.14.142.2:39193/ using document root <none> and password file <none>
I20260926 21:28:15.822219 2020 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:15.822418 2020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:15.822969 2020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:15.826471 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "c01d6792d26f414da4b9b4f2d5dd438e"
format_stamp: "Formatted at 2026-09-26 21:28:15 on dist-test-slave-4d5t"
server_key: "ee4146c7e406912ce41ba8ba8e035571"
server_key_iv: "42a25c5350e124de507d634f3234b61d"
server_key_version: "encryptionkey@0"
I20260926 21:28:15.827335 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "c01d6792d26f414da4b9b4f2d5dd438e"
format_stamp: "Formatted at 2026-09-26 21:28:15 on dist-test-slave-4d5t"
server_key: "ee4146c7e406912ce41ba8ba8e035571"
server_key_iv: "42a25c5350e124de507d634f3234b61d"
server_key_version: "encryptionkey@0"
I20260926 21:28:15.832973 2020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260926 21:28:15.836807 2035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:15.838311 2020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:28:15.838512 2020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "c01d6792d26f414da4b9b4f2d5dd438e"
format_stamp: "Formatted at 2026-09-26 21:28:15 on dist-test-slave-4d5t"
server_key: "ee4146c7e406912ce41ba8ba8e035571"
server_key_iv: "42a25c5350e124de507d634f3234b61d"
server_key_version: "encryptionkey@0"
I20260926 21:28:15.838733 2020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:15.861601 2020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:15.866097 2020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:15.866400 2020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:15.867416 2020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:15.868873 2020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:15.868983 2020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:15.869143 2020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:15.869222 2020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:15.906754 2020 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:38761
I20260926 21:28:15.906841 2147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:38761 every 8 connection(s)
I20260926 21:28:15.908032 2020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:28:15.914857 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2020
I20260926 21:28:15.915154 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:28:15.915700 14904 external_mini_cluster.cc:1428] Setting key c46b6cedce2cbb06ce318290a4297f5b
Sep 26 21:28:15 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458095, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.2@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:15.929108 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:15.910830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:46321 (local address 127.14.142.62:35455)
0926 21:28:15.911295 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:15.911300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:15.912047 (+ 747us) server_negotiation.cc:407] Connection header received
0926 21:28:15.913989 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:15.913997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:15.914090 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:15.914214 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:15.915962 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.916795 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.918669 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.919113 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.919255 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:15.923172 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:15.923178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:15.923181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:15.923220 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:15.925895 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:15.926690 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:15.926700 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:15.926703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:15.926826 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:15.927263 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:15.927269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:15.927273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:15.927575 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:15.927734 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:15.928455 (+ 721us) server_negotiation.cc:299] Negotiation successful
0926 21:28:15.928719 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":257,"threads_started":1}
I20260926 21:28:15.930104 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:15.911169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:35455 (local address 127.14.142.2:46321)
0926 21:28:15.911884 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:15.911921 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:15.913709 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:15.914473 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:15.914481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:15.915080 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:15.915733 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:15.915788 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.916943 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.916949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:15.918518 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:15.918531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:15.919351 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:15.919358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:15.919601 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:15.920327 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:15.920350 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:15.922905 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:15.926063 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:15.926073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:15.926093 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:15.926553 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:15.926941 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:15.926946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:15.926950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:15.927149 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:15.927705 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:15.927714 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:15.928083 (+ 369us) client_negotiation.cc:769] Sending connection context
0926 21:28:15.928367 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 21:28:15.928964 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":275,"threads_started":1}
I20260926 21:28:15.931653 2148 heartbeater.cc:347] Connected to a master server at 127.14.142.62:35455
I20260926 21:28:15.932008 2148 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:15.932934 2148 heartbeater.cc:511] Master 127.14.142.62:35455 requested a full tablet report, sending...
I20260926 21:28:15.934674 1825 ts_manager.cc:194] Registered new tserver with Master: c01d6792d26f414da4b9b4f2d5dd438e (127.14.142.2:38761)
I20260926 21:28:15.935976 1825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.2@KRBTEST.COM'} at 127.14.142.2:46321
WARNING: no policy specified for kudu/127.14.142.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
Entry for principal kudu/127.14.142.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab.
I20260926 21:28:15.980427 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:35455
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:16.328979 2154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:16.329272 2154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:16.329403 2154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:16.337044 2154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:16.337158 2154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:16.337352 2154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:28:16.348225 2154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.3.keytab
--principal=kudu/127.14.142.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:35455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:16.349565 2154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:16.350953 2154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:16.364741 2159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:16.364694 2162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:16.364787 2160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:16.365713 2154 server_base.cc:1034] running on GCE node
I20260926 21:28:16.366371 2154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:16.367522 2154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:16.369055 2154 hybrid_clock.cc:648] HybridClock initialized: now 1790458096368942 us; error 105 us; skew 500 ppm
I20260926 21:28:16.369387 2154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:16 dist-test-slave-4d5t krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458096, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:16.373306 2154 init.cc:377] Logged in from keytab as kudu/127.14.142.3@KRBTEST.COM (short username kudu)
I20260926 21:28:16.375161 2154 webserver.cc:460] Webserver started at http://127.14.142.3:37531/ using document root <none> and password file <none>
I20260926 21:28:16.375946 2154 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:16.376094 2154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:16.376471 2154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:16.379621 2154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "766ad7cf93d34ba083142d8278a12c07"
format_stamp: "Formatted at 2026-09-26 21:28:16 on dist-test-slave-4d5t"
server_key: "ecb5dd1c212ddf67e2d15a223c4bfca3"
server_key_iv: "e5f91370cd7c440796390b8d7384dc24"
server_key_version: "encryptionkey@0"
I20260926 21:28:16.380432 2154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "766ad7cf93d34ba083142d8278a12c07"
format_stamp: "Formatted at 2026-09-26 21:28:16 on dist-test-slave-4d5t"
server_key: "ecb5dd1c212ddf67e2d15a223c4bfca3"
server_key_iv: "e5f91370cd7c440796390b8d7384dc24"
server_key_version: "encryptionkey@0"
I20260926 21:28:16.384748 2014 heartbeater.cc:503] Master 127.14.142.62:35455 was elected leader, sending a full tablet report...
I20260926 21:28:16.386315 2154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 21:28:16.390192 2169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:16.391607 2154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 21:28:16.391834 2154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "766ad7cf93d34ba083142d8278a12c07"
format_stamp: "Formatted at 2026-09-26 21:28:16 on dist-test-slave-4d5t"
server_key: "ecb5dd1c212ddf67e2d15a223c4bfca3"
server_key_iv: "e5f91370cd7c440796390b8d7384dc24"
server_key_version: "encryptionkey@0"
I20260926 21:28:16.392064 2154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:16.417327 2154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:16.422405 2154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:16.422720 2154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:16.423792 2154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:16.425246 2154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:16.425356 2154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:16.425506 2154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:16.425583 2154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:16.462571 2154 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:41241
I20260926 21:28:16.462620 2282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:41241 every 8 connection(s)
I20260926 21:28:16.464039 2154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:28:16.470026 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2154
I20260926 21:28:16.470299 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:28:16.471001 14904 external_mini_cluster.cc:1428] Setting key c69ff7360b07f54dc8fb70081661d689
Sep 26 21:28:16 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458096, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.3@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:16.486413 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.466911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:43735 (local address 127.14.142.62:35455)
0926 21:28:16.467355 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.467360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.468423 (+ 1063us) server_negotiation.cc:407] Connection header received
0926 21:28:16.470616 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.470624 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.470719 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.470854 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:16.472776 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.473893 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.475601 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.476015 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.476161 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.479848 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:16.479853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:16.479856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:16.479891 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:16.482758 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.483564 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.483568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.483570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.483694 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.484164 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.484174 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.484176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.484587 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:16.484758 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.485715 (+ 957us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.485952 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":208,"threads_started":1}
I20260926 21:28:16.487494 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.467447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:35455 (local address 127.14.142.3:43735)
0926 21:28:16.468236 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.468277 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:16.470126 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:16.471108 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:16.471124 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:16.471819 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:16.472512 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.472538 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.474065 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.474074 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.475434 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.475448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.476276 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.476284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.476528 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.477326 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:28:16.477351 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:16.479595 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:16.482924 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:16.482934 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:16.482953 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:16.483412 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:16.483813 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:28:16.483818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:28:16.483821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:28:16.484014 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:28:16.484759 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:16.484770 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:16.485259 (+ 489us) client_negotiation.cc:769] Sending connection context
0926 21:28:16.485569 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 21:28:16.486211 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":313,"threads_started":1}
I20260926 21:28:16.489521 2283 heartbeater.cc:347] Connected to a master server at 127.14.142.62:35455
I20260926 21:28:16.489909 2283 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:16.490692 2283 heartbeater.cc:511] Master 127.14.142.62:35455 requested a full tablet report, sending...
I20260926 21:28:16.492481 1825 ts_manager.cc:194] Registered new tserver with Master: 766ad7cf93d34ba083142d8278a12c07 (127.14.142.3:41241)
I20260926 21:28:16.493389 1825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.14.142.3@KRBTEST.COM'} at 127.14.142.3:43735
I20260926 21:28:16.500416 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:28:16.519542 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.505089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.14.142.62:35455)
0926 21:28:16.505325 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.505335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.505988 (+ 653us) server_negotiation.cc:407] Connection header received
0926 21:28:16.506201 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.506205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.506309 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.506396 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:16.508102 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.508978 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.510560 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.511032 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.511156 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.513882 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:16.513887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:16.513891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:16.513928 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:16.516718 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.517384 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.517389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.517391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.517541 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.517984 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.517991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.517998 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.518351 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:16.518624 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.518975 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.519209 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:28:16.523641 1825 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:28:16.531939 1825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:28:16.571264 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.548020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.14.142.3:41241)
0926 21:28:16.559684 (+ 11664us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.559697 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.560403 (+ 706us) server_negotiation.cc:407] Connection header received
0926 21:28:16.560519 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.560528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.560836 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.561811 (+ 975us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:16.562144 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.563840 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.568449 (+ 4609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.570028 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.570186 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.570446 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.570710 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.570886 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11455,"thread_start_us":11259,"threads_started":1}
I20260926 21:28:16.571368 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.547666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:41241 (local address 127.0.0.1:58958)
0926 21:28:16.559952 (+ 12286us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.559973 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:16.560192 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:16.561113 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:16.561122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:16.561160 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:16.561585 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.561601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.564022 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.564028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.568271 (+ 4243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.568284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.570252 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.570258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.570466 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.570522 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 21:28:16.570663 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 21:28:16.570871 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12157,"mutex_wait_us":4,"thread_start_us":12059,"threads_started":1}
I20260926 21:28:16.573154 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.547259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36694)
0926 21:28:16.557606 (+ 10347us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.557695 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:16.560996 (+ 3301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:16.562555 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:16.562562 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:16.562594 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:16.564649 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.564667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.566336 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.566345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.569054 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.569069 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.572120 (+ 3051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.572128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.572364 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.572413 (+ 49us) client_negotiation.cc:769] Sending connection context
0926 21:28:16.572528 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 21:28:16.572726 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9806,"thread_start_us":12326,"threads_started":1}
I20260926 21:28:16.574098 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.547530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.14.142.1:42099)
0926 21:28:16.561642 (+ 14112us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.561651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.561680 (+ 29us) server_negotiation.cc:407] Connection header received
0926 21:28:16.561783 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.561792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.562038 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.562245 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:16.564939 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.566172 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.569239 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.570780 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.570924 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.571142 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.573542 (+ 2400us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.573754 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13895,"thread_start_us":13714,"threads_started":1}
I20260926 21:28:16.576519 2218 tablet_service.cc:1467] Processing CreateTablet for tablet 7be44574a01449e6bba1f66d82b6f525 (DEFAULT_TABLE table=test-table [id=76cf0c5b6a8d430e96637e751f9cf493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:16.578294 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.561127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.14.142.2:38761)
0926 21:28:16.565025 (+ 3898us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.565031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.567470 (+ 2439us) server_negotiation.cc:407] Connection header received
0926 21:28:16.568196 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.568206 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.568526 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.568787 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:16.569589 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.571390 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.575994 (+ 4604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.577197 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.577358 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.577584 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.577790 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.577962 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3734,"thread_start_us":3579,"threads_started":1}
I20260926 21:28:16.578316 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.561773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38761 (local address 127.0.0.1:33856)
0926 21:28:16.567262 (+ 5489us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.567289 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:16.567869 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:16.568785 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:16.568790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:16.568821 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:16.569397 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.569414 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.571577 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.571582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.575802 (+ 4220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.575817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.577411 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.577417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.577586 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.577626 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 21:28:16.577745 (+ 119us) client_negotiation.cc:240] Negotiation successful
0926 21:28:16.577883 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5308,"mutex_wait_us":30,"thread_start_us":4857,"threads_started":1}
I20260926 21:28:16.578477 2218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be44574a01449e6bba1f66d82b6f525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:16.578405 1949 tablet_service.cc:1467] Processing CreateTablet for tablet 7be44574a01449e6bba1f66d82b6f525 (DEFAULT_TABLE table=test-table [id=76cf0c5b6a8d430e96637e751f9cf493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:16.579982 1949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be44574a01449e6bba1f66d82b6f525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:16.588441 2083 tablet_service.cc:1467] Processing CreateTablet for tablet 7be44574a01449e6bba1f66d82b6f525 (DEFAULT_TABLE table=test-table [id=76cf0c5b6a8d430e96637e751f9cf493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:16.589799 2083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7be44574a01449e6bba1f66d82b6f525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:16.596603 2302 tablet_bootstrap.cc:492] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Bootstrap starting.
I20260926 21:28:16.596493 2303 tablet_bootstrap.cc:492] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Bootstrap starting.
I20260926 21:28:16.601665 2303 tablet_bootstrap.cc:654] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:16.603075 2303 log.cc:824] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:16.604038 2304 tablet_bootstrap.cc:492] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Bootstrap starting.
I20260926 21:28:16.604209 2302 tablet_bootstrap.cc:654] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:16.606175 2302 log.cc:824] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:16.608588 2304 tablet_bootstrap.cc:654] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:16.608850 2303 tablet_bootstrap.cc:492] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: No bootstrap required, opened a new log
I20260926 21:28:16.609213 2303 ts_tablet_manager.cc:1397] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260926 21:28:16.609506 2302 tablet_bootstrap.cc:492] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: No bootstrap required, opened a new log
I20260926 21:28:16.609839 2302 ts_tablet_manager.cc:1397] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.002s
I20260926 21:28:16.609987 2304 log.cc:824] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:16.612689 2304 tablet_bootstrap.cc:492] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: No bootstrap required, opened a new log
I20260926 21:28:16.612962 2304 ts_tablet_manager.cc:1397] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.004s
I20260926 21:28:16.620638 2303 raft_consensus.cc:348] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.620638 2302 raft_consensus.cc:348] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.621143 2303 raft_consensus.cc:374] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:16.621143 2302 raft_consensus.cc:374] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:16.621246 2302 raft_consensus.cc:729] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a47a31839e4a8fb02b78a2a10a6570, State: Initialized, Role: FOLLOWER
I20260926 21:28:16.621277 2303 raft_consensus.cc:729] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766ad7cf93d34ba083142d8278a12c07, State: Initialized, Role: FOLLOWER
I20260926 21:28:16.621770 2304 raft_consensus.cc:348] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.621939 2303 consensus_queue.cc:260] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.621938 2302 consensus_queue.cc:260] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.622150 2304 raft_consensus.cc:374] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:16.622272 2304 raft_consensus.cc:729] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01d6792d26f414da4b9b4f2d5dd438e, State: Initialized, Role: FOLLOWER
I20260926 21:28:16.623004 2304 consensus_queue.cc:260] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.624200 2302 ts_tablet_manager.cc:1428] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260926 21:28:16.625339 2283 heartbeater.cc:503] Master 127.14.142.62:35455 was elected leader, sending a full tablet report...
I20260926 21:28:16.625978 2303 ts_tablet_manager.cc:1428] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
I20260926 21:28:16.627074 2148 heartbeater.cc:503] Master 127.14.142.62:35455 was elected leader, sending a full tablet report...
I20260926 21:28:16.629031 2304 ts_tablet_manager.cc:1428] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
W20260926 21:28:16.660343 2149 tablet.cc:2369] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:16.662448 2309 raft_consensus.cc:482] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:16.662761 2309 raft_consensus.cc:504] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.664422 2309 leader_election.cc:288] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89a47a31839e4a8fb02b78a2a10a6570 (127.14.142.1:42099), c01d6792d26f414da4b9b4f2d5dd438e (127.14.142.2:38761)
I20260926 21:28:16.671037 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.664943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:47187 (local address 127.14.142.1:42099)
0926 21:28:16.665210 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.665217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.665318 (+ 101us) server_negotiation.cc:407] Connection header received
0926 21:28:16.665588 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.665594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.665677 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.665771 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:16.666219 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.666918 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.668579 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.669639 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.669829 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.669890 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.670620 (+ 730us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.670791 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260926 21:28:16.671093 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.664856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.14.142.3:47187)
0926 21:28:16.665191 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.665213 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:16.665422 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:16.665814 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:16.665823 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:16.665851 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:16.666057 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.666066 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.667064 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.667072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.668381 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.668397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.669962 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.669972 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.670259 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.670313 (+ 54us) client_negotiation.cc:769] Sending connection context
0926 21:28:16.670494 (+ 181us) client_negotiation.cc:240] Negotiation successful
0926 21:28:16.670704 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":4}
I20260926 21:28:16.672484 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.665299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:38761 (local address 127.14.142.3:37307)
0926 21:28:16.666017 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.666031 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:16.666185 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:16.666500 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:16.666504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:16.666527 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:16.666837 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.666845 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.667802 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.667806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.669490 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:16.669502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.671572 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.671581 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:16.671828 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.671865 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 21:28:16.671988 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 21:28:16.672160 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":20,"thread_start_us":402,"threads_started":1}
I20260926 21:28:16.672959 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.665463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:37307 (local address 127.14.142.2:38761)
0926 21:28:16.665718 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.665726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.666123 (+ 397us) server_negotiation.cc:407] Connection header received
0926 21:28:16.666317 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.666322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.666386 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.666489 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:16.666978 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.667644 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.669637 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.671288 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.671507 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.671605 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.672124 (+ 519us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.672433 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260926 21:28:16.673075 1969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be44574a01449e6bba1f66d82b6f525" candidate_uuid: "766ad7cf93d34ba083142d8278a12c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a47a31839e4a8fb02b78a2a10a6570" is_pre_election: true
I20260926 21:28:16.673914 1969 raft_consensus.cc:2454] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766ad7cf93d34ba083142d8278a12c07 in term 0.
I20260926 21:28:16.674078 2103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be44574a01449e6bba1f66d82b6f525" candidate_uuid: "766ad7cf93d34ba083142d8278a12c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" is_pre_election: true
I20260926 21:28:16.674566 2103 raft_consensus.cc:2454] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766ad7cf93d34ba083142d8278a12c07 in term 0.
I20260926 21:28:16.674822 2172 leader_election.cc:302] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766ad7cf93d34ba083142d8278a12c07, 89a47a31839e4a8fb02b78a2a10a6570; no voters:
I20260926 21:28:16.675302 2309 raft_consensus.cc:2781] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:28:16.675498 2309 raft_consensus.cc:482] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:28:16.675642 2309 raft_consensus.cc:3037] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:16.678207 2309 raft_consensus.cc:504] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.678821 2309 leader_election.cc:288] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [CANDIDATE]: Term 1 election: Requested vote from peers 89a47a31839e4a8fb02b78a2a10a6570 (127.14.142.1:42099), c01d6792d26f414da4b9b4f2d5dd438e (127.14.142.2:38761)
I20260926 21:28:16.679780 1969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be44574a01449e6bba1f66d82b6f525" candidate_uuid: "766ad7cf93d34ba083142d8278a12c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89a47a31839e4a8fb02b78a2a10a6570"
I20260926 21:28:16.679877 2103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7be44574a01449e6bba1f66d82b6f525" candidate_uuid: "766ad7cf93d34ba083142d8278a12c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01d6792d26f414da4b9b4f2d5dd438e"
I20260926 21:28:16.680001 1969 raft_consensus.cc:3037] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:16.680229 2103 raft_consensus.cc:3037] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:16.682451 1969 raft_consensus.cc:2454] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766ad7cf93d34ba083142d8278a12c07 in term 1.
I20260926 21:28:16.683128 2103 raft_consensus.cc:2454] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766ad7cf93d34ba083142d8278a12c07 in term 1.
I20260926 21:28:16.683122 2172 leader_election.cc:302] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766ad7cf93d34ba083142d8278a12c07, 89a47a31839e4a8fb02b78a2a10a6570; no voters:
I20260926 21:28:16.683547 2309 raft_consensus.cc:2781] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:16.684171 2309 raft_consensus.cc:686] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 1 LEADER]: Becoming Leader. State: Replica: 766ad7cf93d34ba083142d8278a12c07, State: Running, Role: LEADER
I20260926 21:28:16.684886 2309 consensus_queue.cc:237] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } }
I20260926 21:28:16.690635 1824 catalog_manager.cc:5495] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 766ad7cf93d34ba083142d8278a12c07 (127.14.142.3). New cstate: current_term: 1 leader_uuid: "766ad7cf93d34ba083142d8278a12c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "766ad7cf93d34ba083142d8278a12c07" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 41241 } health_report { overall_health: HEALTHY } } }
W20260926 21:28:16.716173 2284 tablet.cc:2369] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:16.739603 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.733687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58972 (local address 127.14.142.3:41241)
0926 21:28:16.733968 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.733973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.733992 (+ 19us) server_negotiation.cc:407] Connection header received
0926 21:28:16.734176 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.734180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.734276 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.734414 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:28:16.734916 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.735726 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.737141 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.737577 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.737708 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.738345 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:28:16.738872 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:28:16.739019 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.739190 (+ 171us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.739260 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 21:28:16.749413 2103 raft_consensus.cc:1261] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 1 FOLLOWER]: Refusing update from remote peer 766ad7cf93d34ba083142d8278a12c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:16.749408 1969 raft_consensus.cc:1261] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 1 FOLLOWER]: Refusing update from remote peer 766ad7cf93d34ba083142d8278a12c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:16.750486 2312 consensus_queue.cc:1035] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89a47a31839e4a8fb02b78a2a10a6570" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 42099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:16.750897 2309 consensus_queue.cc:1035] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01d6792d26f414da4b9b4f2d5dd438e" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 38761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:16.764632 2315 mvcc.cc:205] Tried to move back new op lower bound from 7333716364272066560 to 7333716364026318848. Current Snapshot: MvccSnapshot[applied={T|T < 7333716364272066560}]
I20260926 21:28:16.766757 2316 mvcc.cc:205] Tried to move back new op lower bound from 7333716364272066560 to 7333716364026318848. Current Snapshot: MvccSnapshot[applied={T|T < 7333716364272066560}]
I20260926 21:28:16.779044 1825 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:16.779340 1825 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:16.783198 1825 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02cc0403f86e47b8ac341694f25b48f7: Sending DeleteTablet for 3 replicas of tablet 7be44574a01449e6bba1f66d82b6f525
I20260926 21:28:16.784574 2083 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be44574a01449e6bba1f66d82b6f525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:16 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:33856
I20260926 21:28:16.784543 1949 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be44574a01449e6bba1f66d82b6f525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:16 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:36694
I20260926 21:28:16.784770 2218 tablet_service.cc:1514] Processing DeleteTablet for tablet 7be44574a01449e6bba1f66d82b6f525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:16 UTC) from {username='kudu', principal='kudu/127.14.142.62@KRBTEST.COM'} at 127.0.0.1:58958
I20260926 21:28:16.785923 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1886
I20260926 21:28:16.805477 2326 tablet_replica.cc:330] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: stopping tablet replica
I20260926 21:28:16.806056 2326 raft_consensus.cc:2229] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:16.806689 2326 raft_consensus.cc:2258] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:16.807755 2327 tablet_replica.cc:330] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: stopping tablet replica
I20260926 21:28:16.808166 2325 tablet_replica.cc:330] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: stopping tablet replica
I20260926 21:28:16.808223 2327 raft_consensus.cc:2229] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:28:16.808705 2325 raft_consensus.cc:2229] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:16.809047 2327 raft_consensus.cc:2258] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:16.809340 2325 raft_consensus.cc:2258] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:16.810647 2326 ts_tablet_manager.cc:1905] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:16.811218 2327 ts_tablet_manager.cc:1905] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:16.811285 2325 ts_tablet_manager.cc:1905] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 21:28:16 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.1@KRBTEST.COM
I20260926 21:28:16.819121 2325 ts_tablet_manager.cc:1918] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:16.819414 2325 log.cc:1196] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/7be44574a01449e6bba1f66d82b6f525
I20260926 21:28:16.819648 2327 ts_tablet_manager.cc:1918] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:16.819857 2327 log.cc:1196] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/7be44574a01449e6bba1f66d82b6f525
I20260926 21:28:16.820228 2325 ts_tablet_manager.cc:1939] T 7be44574a01449e6bba1f66d82b6f525 P c01d6792d26f414da4b9b4f2d5dd438e: Deleting consensus metadata
I20260926 21:28:16.820459 2327 ts_tablet_manager.cc:1939] T 7be44574a01449e6bba1f66d82b6f525 P 766ad7cf93d34ba083142d8278a12c07: Deleting consensus metadata
I20260926 21:28:16.822474 1812 catalog_manager.cc:4841] TS 766ad7cf93d34ba083142d8278a12c07 (127.14.142.3:41241): tablet 7be44574a01449e6bba1f66d82b6f525 (table test-table [id=76cf0c5b6a8d430e96637e751f9cf493]) successfully deleted
I20260926 21:28:16.822855 1811 catalog_manager.cc:4841] TS c01d6792d26f414da4b9b4f2d5dd438e (127.14.142.2:38761): tablet 7be44574a01449e6bba1f66d82b6f525 (table test-table [id=76cf0c5b6a8d430e96637e751f9cf493]) successfully deleted
I20260926 21:28:16.827576 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.787652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.14.142.1:42099)
0926 21:28:16.787902 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.787912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.799429 (+ 11517us) server_negotiation.cc:407] Connection header received
0926 21:28:16.799956 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.799964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.800065 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.800169 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:16.802214 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.803563 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.805891 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.806689 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.806825 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.819074 (+ 12249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:16.819096 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:16.819116 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:16.819176 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:16.823603 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.824641 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.824651 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.824655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.824884 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.825381 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.825387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.825391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.825932 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:16.826291 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.826639 (+ 348us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.826921 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:28:16.828073 2326 ts_tablet_manager.cc:1918] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:16.828303 2326 log.cc:1196] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/7be44574a01449e6bba1f66d82b6f525
I20260926 21:28:16.828389 2009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:16.933735 2326 ts_tablet_manager.cc:1939] T 7be44574a01449e6bba1f66d82b6f525 P 89a47a31839e4a8fb02b78a2a10a6570: Deleting consensus metadata
I20260926 21:28:16.934361 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1886
W20260926 21:28:16.951261 1811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.142.1:42099 (error 108)
I20260926 21:28:16.951538 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2020
I20260926 21:28:16.953668 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.953192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36702)
0926 21:28:16.953397 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:16.953512 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260926 21:28:16.953913 1811 catalog_manager.cc:4584] TS 89a47a31839e4a8fb02b78a2a10a6570 (127.14.142.1:42099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7be44574a01449e6bba1f66d82b6f525: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Sep 26 21:28:16 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.2@KRBTEST.COM
I20260926 21:28:16.969343 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:16.952853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.14.142.2:38761)
0926 21:28:16.953058 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:16.953066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:16.953089 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:28:16.953157 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:16.953163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:16.953256 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:16.953362 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:16.955243 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.956044 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.957523 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:16.957933 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:16.958081 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:16.962714 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:16.962731 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:16.962749 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:16.962792 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:16.965882 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.966649 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.966656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.966659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.966791 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:16.967200 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:16.967205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:16.967206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:16.967932 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:16.968283 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:16.968691 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 21:28:16.968942 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:28:16.970181 2143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:17.007614 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.007049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36710)
0926 21:28:17.007278 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.007397 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 21:28:17.008984 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.008518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36712)
0926 21:28:17.008746 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.008852 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260926 21:28:17.068514 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2020
I20260926 21:28:17.069701 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.069122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36718)
0926 21:28:17.069366 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.069480 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 21:28:17.071048 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.070555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36732)
0926 21:28:17.070787 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.070878 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 21:28:17.087339 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2154
Sep 26 21:28:17 dist-test-slave-4d5t krb5kdc[1780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458094, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.3@KRBTEST.COM
I20260926 21:28:17.108023 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.088793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.14.142.3:41241)
0926 21:28:17.089006 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:17.089013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:17.089035 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:28:17.089103 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:17.089109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:17.089231 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:17.089342 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:17.090728 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:17.091970 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:17.093441 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:17.094162 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:17.094328 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:17.100027 (+ 5699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:17.100043 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:17.100063 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:17.100121 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:17.104622 (+ 4501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:17.105544 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:17.105552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:17.105554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:17.105712 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:17.106134 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:17.106144 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:17.106146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:17.106606 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:17.106995 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:17.107368 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 21:28:17.107604 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:28:17.109046 2278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:17.151984 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.151448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36742)
0926 21:28:17.151739 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.151838 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260926 21:28:17.153178 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.152795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36756)
0926 21:28:17.152997 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.153068 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 21:28:17.238919 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2154
I20260926 21:28:17.259449 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 1793
I20260926 21:28:17.260869 1855 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:17.374770 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.374102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:42099 (local address 127.0.0.1:36762)
0926 21:28:17.374289 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:17.374381 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:42099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 21:28:17.374926 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 1793
2026-09-26T21:28:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4644 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T21:28:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:17Z Disabled control of system clock
I20260926 21:28:17.460906 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38187
--webserver_interface=127.14.142.62
--webserver_port=0
--builtin_ntp_servers=127.14.142.20:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:38187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 21:28:17.827143 2337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:17.827843 2337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:17.827972 2337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:17.836328 2337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:17.836483 2337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:17.836525 2337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:17.836601 2337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:17.836648 2337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:17.847019 2337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:38187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:17.849216 2337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:17.851567 2337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:17.865757 2343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:17.865777 2345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:17.866031 2342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:17.866668 2337 server_base.cc:1034] running on GCE node
I20260926 21:28:17.867564 2337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:17.869318 2337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:17.870550 2337 hybrid_clock.cc:648] HybridClock initialized: now 1790458097870504 us; error 35 us; skew 500 ppm
I20260926 21:28:17.870913 2337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:17.873639 2337 webserver.cc:460] Webserver started at http://127.14.142.62:34495/ using document root <none> and password file <none>
I20260926 21:28:17.874424 2337 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:17.874524 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:17.874811 2337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:17.878224 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "097b05ff4f3145d5bef988d7f3265eaa"
format_stamp: "Formatted at 2026-09-26 21:28:17 on dist-test-slave-4d5t"
server_key: "a721fbafe8a868cd264c11ed4f3a8162"
server_key_iv: "14ea838e65a37f235daac073b5eb1991"
server_key_version: "encryptionkey@0"
I20260926 21:28:17.879196 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "097b05ff4f3145d5bef988d7f3265eaa"
format_stamp: "Formatted at 2026-09-26 21:28:17 on dist-test-slave-4d5t"
server_key: "a721fbafe8a868cd264c11ed4f3a8162"
server_key_iv: "14ea838e65a37f235daac073b5eb1991"
server_key_version: "encryptionkey@0"
I20260926 21:28:17.885763 2337 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:28:17.890131 2351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:17.891769 2337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 21:28:17.891945 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "097b05ff4f3145d5bef988d7f3265eaa"
format_stamp: "Formatted at 2026-09-26 21:28:17 on dist-test-slave-4d5t"
server_key: "a721fbafe8a868cd264c11ed4f3a8162"
server_key_iv: "14ea838e65a37f235daac073b5eb1991"
server_key_version: "encryptionkey@0"
I20260926 21:28:17.892155 2337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:17.918236 2337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:17.919166 2337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:17.919535 2337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:17.940281 2337 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:38187
I20260926 21:28:17.940310 2402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:38187 every 8 connection(s)
I20260926 21:28:17.941812 2337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:17.945927 2403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:17.950336 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2337
I20260926 21:28:17.950500 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:17.950987 14904 external_mini_cluster.cc:1428] Setting key 8d0bd185c28242e70c663bc76510ab48
I20260926 21:28:17.957794 2403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa: Bootstrap starting.
I20260926 21:28:17.961619 2403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:17.962975 2403 log.cc:824] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:17.966374 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:17.953140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.14.142.62:38187)
0926 21:28:17.954260 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:17.954282 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:17.954328 (+ 46us) server_negotiation.cc:407] Connection header received
0926 21:28:17.956330 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:17.956356 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:17.956934 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:17.957275 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:17.958033 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:17.959394 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:17.961311 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:17.962124 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:17.962271 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:17.963599 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:17.963615 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:17.963641 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:17.964491 (+ 850us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:17.964644 (+ 153us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:17.964668 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:17.964762 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:17.965096 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 21:28:17.965455 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":456,"threads_started":1}
I20260926 21:28:17.966616 2403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa: No bootstrap required, opened a new log
I20260926 21:28:17.974927 2403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097b05ff4f3145d5bef988d7f3265eaa" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38187 } }
I20260926 21:28:17.975528 2403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:17.975662 2403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 097b05ff4f3145d5bef988d7f3265eaa, State: Initialized, Role: FOLLOWER
I20260926 21:28:17.976404 2403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097b05ff4f3145d5bef988d7f3265eaa" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38187 } }
I20260926 21:28:17.976842 2403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:17.977046 2403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:17.977248 2403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:17.979964 2403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097b05ff4f3145d5bef988d7f3265eaa" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38187 } }
I20260926 21:28:17.980638 2403 leader_election.cc:302] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 097b05ff4f3145d5bef988d7f3265eaa; no voters:
I20260926 21:28:17.981339 2403 leader_election.cc:288] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:17.981518 2408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:17.982239 2408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [term 1 LEADER]: Becoming Leader. State: Replica: 097b05ff4f3145d5bef988d7f3265eaa, State: Running, Role: LEADER
I20260926 21:28:17.982785 2408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097b05ff4f3145d5bef988d7f3265eaa" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38187 } }
I20260926 21:28:17.983493 2403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:17.985917 2410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 097b05ff4f3145d5bef988d7f3265eaa. Latest consensus state: current_term: 1 leader_uuid: "097b05ff4f3145d5bef988d7f3265eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097b05ff4f3145d5bef988d7f3265eaa" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38187 } } }
I20260926 21:28:17.985847 2409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "097b05ff4f3145d5bef988d7f3265eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097b05ff4f3145d5bef988d7f3265eaa" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38187 } } }
I20260926 21:28:17.986327 2410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:17.986418 2409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:17.993848 2416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:17.999286 2416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:18.008440 2416 catalog_manager.cc:1347] Generated new cluster ID: 65646a8d145448a498e7d19e82004246
I20260926 21:28:18.008596 2416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:18.020121 2416 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:18.021965 2416 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:18.045899 2416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa: Generated new TSK 0
I20260926 21:28:18.046963 2416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:18.059937 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.1:0
--local_ip_for_outbound_sockets=127.14.142.1
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38187
--builtin_ntp_servers=127.14.142.20:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 21:28:18.414927 2427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:18.415246 2427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:18.415341 2427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:18.422909 2427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:18.423033 2427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:18.423206 2427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.1
I20260926 21:28:18.432941 2427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.14.142.1
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.1
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:18.434405 2427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:18.436290 2427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:18.449993 2432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:18.450034 2435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:18.449951 2433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:18.451208 2427 server_base.cc:1034] running on GCE node
I20260926 21:28:18.451867 2427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:18.453012 2427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:18.454267 2427 hybrid_clock.cc:648] HybridClock initialized: now 1790458098454214 us; error 44 us; skew 500 ppm
I20260926 21:28:18.454639 2427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:18.457401 2427 webserver.cc:460] Webserver started at http://127.14.142.1:46213/ using document root <none> and password file <none>
I20260926 21:28:18.458123 2427 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:18.458262 2427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:18.458595 2427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:18.461776 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data/instance:
uuid: "7303d05b71cb4030a193516d4bf58f1a"
format_stamp: "Formatted at 2026-09-26 21:28:18 on dist-test-slave-4d5t"
server_key: "2b1b03970bcf394e78746a72d7b31126"
server_key_iv: "992484f45fe8ac0c62732a8f271b8cb3"
server_key_version: "encryptionkey@0"
I20260926 21:28:18.462601 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance:
uuid: "7303d05b71cb4030a193516d4bf58f1a"
format_stamp: "Formatted at 2026-09-26 21:28:18 on dist-test-slave-4d5t"
server_key: "2b1b03970bcf394e78746a72d7b31126"
server_key_iv: "992484f45fe8ac0c62732a8f271b8cb3"
server_key_version: "encryptionkey@0"
I20260926 21:28:18.468185 2427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 21:28:18.471985 2441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:18.473635 2427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 21:28:18.473838 2427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal
uuid: "7303d05b71cb4030a193516d4bf58f1a"
format_stamp: "Formatted at 2026-09-26 21:28:18 on dist-test-slave-4d5t"
server_key: "2b1b03970bcf394e78746a72d7b31126"
server_key_iv: "992484f45fe8ac0c62732a8f271b8cb3"
server_key_version: "encryptionkey@0"
I20260926 21:28:18.474118 2427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:18.510018 2427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:18.510946 2427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:18.511276 2427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:18.512393 2427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:18.514073 2427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:18.514204 2427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:18.514348 2427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:18.514444 2427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:18.552841 2427 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.1:36787
I20260926 21:28:18.552929 2553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.1:36787 every 8 connection(s)
I20260926 21:28:18.554221 2427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/data/info.pb
I20260926 21:28:18.558323 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2427
I20260926 21:28:18.558521 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal/instance
I20260926 21:28:18.558965 14904 external_mini_cluster.cc:1428] Setting key 013129bd21e51364525e4058fd993b0c
I20260926 21:28:18.561779 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.2:0
--local_ip_for_outbound_sockets=127.14.142.2
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38187
--builtin_ntp_servers=127.14.142.20:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 21:28:18.571662 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:18.557993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:56637 (local address 127.14.142.62:38187)
0926 21:28:18.558555 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:18.558563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:18.560961 (+ 2398us) server_negotiation.cc:407] Connection header received
0926 21:28:18.563151 (+ 2190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:18.563160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:18.563261 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:18.563413 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:18.565092 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:18.566240 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:18.568163 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:18.568904 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:18.569052 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:18.570763 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:18.570768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:18.570771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:18.570791 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:18.570831 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:18.570852 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:18.570933 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:18.571270 (+ 337us) server_negotiation.cc:299] Negotiation successful
0926 21:28:18.571361 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":257,"threads_started":1}
I20260926 21:28:18.573448 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:18.558812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38187 (local address 127.14.142.1:56637)
0926 21:28:18.560394 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:18.560470 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:18.562814 (+ 2344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:18.563806 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:18.563816 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:18.563934 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:18.564885 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:18.564908 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:18.566531 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:18.566538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:18.567977 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:18.567995 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:18.569197 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:18.569203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:18.569461 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:18.570309 (+ 848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:18.570312 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:18.570359 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:18.570507 (+ 148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:18.570967 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:18.570982 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:18.570994 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 21:28:18.571230 (+ 236us) client_negotiation.cc:240] Negotiation successful
0926 21:28:18.571791 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":428,"threads_started":1}
I20260926 21:28:18.575199 2554 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38187
I20260926 21:28:18.575567 2554 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:18.576403 2554 heartbeater.cc:511] Master 127.14.142.62:38187 requested a full tablet report, sending...
I20260926 21:28:18.578770 2368 ts_manager.cc:194] Registered new tserver with Master: 7303d05b71cb4030a193516d4bf58f1a (127.14.142.1:36787)
I20260926 21:28:18.580292 2368 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.1:56637
W20260926 21:28:18.913543 2558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:18.913833 2558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:18.913995 2558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:18.921861 2558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:18.921990 2558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:18.922204 2558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.2
I20260926 21:28:18.931864 2558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.14.142.2
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.2
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:18.933888 2558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:18.937165 2558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:18.954594 2566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:18.954531 2564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:18.954882 2563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:18.958690 2558 server_base.cc:1034] running on GCE node
I20260926 21:28:18.959447 2558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:18.960517 2558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:18.961839 2558 hybrid_clock.cc:648] HybridClock initialized: now 1790458098961773 us; error 63 us; skew 500 ppm
I20260926 21:28:18.962227 2558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:18.964993 2558 webserver.cc:460] Webserver started at http://127.14.142.2:34995/ using document root <none> and password file <none>
I20260926 21:28:18.965767 2558 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:18.965914 2558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:18.966233 2558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:18.969848 2558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data/instance:
uuid: "fd6996953980436b9c658f00bf6b1ecb"
format_stamp: "Formatted at 2026-09-26 21:28:18 on dist-test-slave-4d5t"
server_key: "7e904c4c486ab9b65b964c8988f9810d"
server_key_iv: "ad75dac035504b9d36ee9b88d0f6343a"
server_key_version: "encryptionkey@0"
I20260926 21:28:18.970710 2558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance:
uuid: "fd6996953980436b9c658f00bf6b1ecb"
format_stamp: "Formatted at 2026-09-26 21:28:18 on dist-test-slave-4d5t"
server_key: "7e904c4c486ab9b65b964c8988f9810d"
server_key_iv: "ad75dac035504b9d36ee9b88d0f6343a"
server_key_version: "encryptionkey@0"
I20260926 21:28:18.976082 2558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 21:28:18.981026 2572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:18.983336 2558 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260926 21:28:18.983697 2558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal
uuid: "fd6996953980436b9c658f00bf6b1ecb"
format_stamp: "Formatted at 2026-09-26 21:28:18 on dist-test-slave-4d5t"
server_key: "7e904c4c486ab9b65b964c8988f9810d"
server_key_iv: "ad75dac035504b9d36ee9b88d0f6343a"
server_key_version: "encryptionkey@0"
I20260926 21:28:18.983997 2558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:19.009227 2558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:19.010090 2558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:19.010399 2558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:19.011425 2558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:19.013037 2558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:19.013173 2558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:19.013304 2558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:19.013401 2558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:19.065855 2558 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.2:33159
I20260926 21:28:19.065900 2685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.2:33159 every 8 connection(s)
I20260926 21:28:19.067770 2558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/data/info.pb
I20260926 21:28:19.074512 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2558
I20260926 21:28:19.074700 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal/instance
I20260926 21:28:19.075276 14904 external_mini_cluster.cc:1428] Setting key 54ba66666240939c71bc66a3a2d3ab27
I20260926 21:28:19.079326 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.14.142.3:0
--local_ip_for_outbound_sockets=127.14.142.3
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38187
--builtin_ntp_servers=127.14.142.20:46627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 21:28:19.085809 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.072908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.2:44361 (local address 127.14.142.62:38187)
0926 21:28:19.077609 (+ 4701us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.077615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.077636 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:28:19.077893 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.077899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.077983 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.078123 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:19.079383 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.080250 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.082028 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.082462 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.082612 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.084835 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:19.084851 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:19.084862 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:19.084884 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:19.084927 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:19.084940 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:19.085040 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.085305 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.085412 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4569,"thread_start_us":7769,"threads_started":1}
I20260926 21:28:19.087091 2688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.072135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38187 (local address 127.14.142.2:44361)
0926 21:28:19.074319 (+ 2184us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.074377 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.077577 (+ 3200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.078490 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.078498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.078604 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:19.079201 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.079217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.080417 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.080422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.081873 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.081888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.082719 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.082727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.082964 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.084005 (+ 1041us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:19.084010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:19.084361 (+ 351us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:19.084543 (+ 182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:19.085042 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:19.085055 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:19.085066 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.085280 (+ 214us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.085748 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"thread_start_us":714,"threads_started":1}
I20260926 21:28:19.088847 2686 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38187
I20260926 21:28:19.089219 2686 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:19.090286 2686 heartbeater.cc:511] Master 127.14.142.62:38187 requested a full tablet report, sending...
I20260926 21:28:19.091889 2368 ts_manager.cc:194] Registered new tserver with Master: fd6996953980436b9c658f00bf6b1ecb (127.14.142.2:33159)
I20260926 21:28:19.093318 2368 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.2:44361
W20260926 21:28:19.456235 2690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:19.456648 2690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:19.456816 2690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:19.465214 2690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:19.465369 2690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:19.465554 2690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.14.142.3
I20260926 21:28:19.475793 2690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:46627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.14.142.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.14.142.3
--webserver_port=0
--tserver_master_addrs=127.14.142.62:38187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.14.142.3
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:19.477348 2690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:19.479317 2690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:19.494238 2696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:19.494268 2695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:19.494920 2698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:19.495468 2690 server_base.cc:1034] running on GCE node
I20260926 21:28:19.496134 2690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:19.497203 2690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:19.498449 2690 hybrid_clock.cc:648] HybridClock initialized: now 1790458099498398 us; error 39 us; skew 500 ppm
I20260926 21:28:19.498842 2690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 21:28:19.501428 2690 webserver.cc:460] Webserver started at http://127.14.142.3:33485/ using document root <none> and password file <none>
I20260926 21:28:19.502276 2690 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:19.502382 2690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:19.502737 2690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:19.505889 2690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data/instance:
uuid: "74b5ee9bf4154b3c98c13c02cead1e39"
format_stamp: "Formatted at 2026-09-26 21:28:19 on dist-test-slave-4d5t"
server_key: "f08f6ab9963f3048c0dba49e044647d5"
server_key_iv: "52e66dbaa0c7e0c6214cebb17e737a2a"
server_key_version: "encryptionkey@0"
I20260926 21:28:19.506747 2690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance:
uuid: "74b5ee9bf4154b3c98c13c02cead1e39"
format_stamp: "Formatted at 2026-09-26 21:28:19 on dist-test-slave-4d5t"
server_key: "f08f6ab9963f3048c0dba49e044647d5"
server_key_iv: "52e66dbaa0c7e0c6214cebb17e737a2a"
server_key_version: "encryptionkey@0"
I20260926 21:28:19.512360 2690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:28:19.516511 2704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:19.518118 2690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:28:19.518319 2690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal
uuid: "74b5ee9bf4154b3c98c13c02cead1e39"
format_stamp: "Formatted at 2026-09-26 21:28:19 on dist-test-slave-4d5t"
server_key: "f08f6ab9963f3048c0dba49e044647d5"
server_key_iv: "52e66dbaa0c7e0c6214cebb17e737a2a"
server_key_version: "encryptionkey@0"
I20260926 21:28:19.518543 2690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:19.540522 2690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:19.541520 2690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:19.541859 2690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:19.542934 2690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 21:28:19.544826 2690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 21:28:19.544922 2690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:19.545063 2690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 21:28:19.545120 2690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:19.582682 2690 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.3:45639
I20260926 21:28:19.582805 2816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.3:45639 every 8 connection(s)
I20260926 21:28:19.583705 2554 heartbeater.cc:503] Master 127.14.142.62:38187 was elected leader, sending a full tablet report...
I20260926 21:28:19.584221 2690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/data/info.pb
I20260926 21:28:19.588276 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2690
I20260926 21:28:19.588518 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal/instance
I20260926 21:28:19.589270 14904 external_mini_cluster.cc:1428] Setting key daa54093bc151a62eaf18eb42e6c6dff
I20260926 21:28:19.600021 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.587128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.3:54005 (local address 127.14.142.62:38187)
0926 21:28:19.587632 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.587637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.589840 (+ 2203us) server_negotiation.cc:407] Connection header received
0926 21:28:19.592453 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.592458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.592531 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.592704 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:19.594117 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.594912 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.596704 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.597299 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.597448 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.599024 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:19.599030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:19.599034 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:19.599057 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:19.599109 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:19.599124 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:19.599231 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.599508 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.599645 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":271,"threads_started":1}
I20260926 21:28:19.601063 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.588191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:38187 (local address 127.14.142.3:54005)
0926 21:28:19.589641 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.589692 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.591977 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.593133 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.593143 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.593250 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:28:19.593952 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.593972 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.595058 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.595062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.596309 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.596322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.597602 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.597610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.597852 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.598631 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 21:28:19.598634 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:28:19.598677 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 21:28:19.598768 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:28:19.599230 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:28:19.599238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:28:19.599244 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.599397 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.599750 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":492,"threads_started":1}
I20260926 21:28:19.603050 2817 heartbeater.cc:347] Connected to a master server at 127.14.142.62:38187
I20260926 21:28:19.603423 2817 heartbeater.cc:464] Registering TS with master...
I20260926 21:28:19.604504 2817 heartbeater.cc:511] Master 127.14.142.62:38187 requested a full tablet report, sending...
I20260926 21:28:19.606397 2368 ts_manager.cc:194] Registered new tserver with Master: 74b5ee9bf4154b3c98c13c02cead1e39 (127.14.142.3:45639)
I20260926 21:28:19.607316 2368 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.14.142.3:54005
I20260926 21:28:19.618592 14904 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 21:28:19.629997 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.623171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.14.142.62:38187)
0926 21:28:19.623492 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.623538 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.623697 (+ 159us) server_negotiation.cc:407] Connection header received
0926 21:28:19.623914 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.623919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.623991 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.624116 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:19.624711 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.625817 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.627265 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.627960 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.628108 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.629236 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:19.629242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:19.629247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:19.629276 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:19.629306 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:19.629318 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:19.629416 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.629570 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.629676 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 21:28:19.635512 2368 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 21:28:19.643932 2368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 21:28:19.673398 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.657704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53078)
0926 21:28:19.661327 (+ 3623us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.661370 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.661657 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.662872 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.662884 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.662950 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:19.663494 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.663513 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.665911 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.665917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.669050 (+ 3133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.669068 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.672150 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.672158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.672391 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.672462 (+ 71us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.672653 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.672910 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3288,"mutex_wait_us":45,"thread_start_us":3069,"threads_started":1}
I20260926 21:28:19.674743 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.657985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53078 (local address 127.14.142.1:36787)
0926 21:28:19.660752 (+ 2767us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.660762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.661486 (+ 724us) server_negotiation.cc:407] Connection header received
0926 21:28:19.661971 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.661979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.662302 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.662504 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:19.663741 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.665685 (+ 1944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.669339 (+ 3654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.671445 (+ 2106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.671633 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.671991 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.674091 (+ 2100us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.674301 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2535,"thread_start_us":2372,"threads_started":1}
I20260926 21:28:19.676537 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.658578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.14.142.2:33159)
0926 21:28:19.660171 (+ 1593us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.660186 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.662657 (+ 2471us) server_negotiation.cc:407] Connection header received
0926 21:28:19.662988 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.662997 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.663252 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.663444 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:19.664443 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.666084 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.672806 (+ 6722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.674507 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.674709 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.675005 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.675869 (+ 864us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.676114 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1333,"thread_start_us":1162,"threads_started":1}
I20260926 21:28:19.677160 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.657956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33159 (local address 127.0.0.1:40094)
0926 21:28:19.662528 (+ 4572us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.662548 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.662777 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.663761 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.663766 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.663794 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:19.664243 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.664256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.668163 (+ 3907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.668182 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.670691 (+ 2509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.670713 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.675174 (+ 4461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.675183 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.675391 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.675429 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.675799 (+ 370us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.676030 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4418,"mutex_wait_us":1,"thread_start_us":4268,"threads_started":1}
I20260926 21:28:19.679677 2489 tablet_service.cc:1467] Processing CreateTablet for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc (DEFAULT_TABLE table=test-table [id=9950cfb1aa764935b807687ca27a5a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:19.680859 2620 tablet_service.cc:1467] Processing CreateTablet for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc (DEFAULT_TABLE table=test-table [id=9950cfb1aa764935b807687ca27a5a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:19.681391 2489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:19.682161 2620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:19.684582 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.662586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:45639 (local address 127.0.0.1:56400)
0926 21:28:19.673715 (+ 11129us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.673738 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.673874 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.676089 (+ 2215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.676097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.676130 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:19.676388 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.676401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.679422 (+ 3021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.679430 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.681382 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.681404 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.683515 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.683522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.683775 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.683822 (+ 47us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.683913 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.684124 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11000,"thread_start_us":10995,"threads_started":1}
I20260926 21:28:19.686000 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.662394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.14.142.3:45639)
0926 21:28:19.675346 (+ 12952us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.675356 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.675397 (+ 41us) server_negotiation.cc:407] Connection header received
0926 21:28:19.675517 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.675525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.675837 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.676834 (+ 997us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:19.677153 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.679211 (+ 2058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.681601 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.683253 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.684859 (+ 1606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.685410 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.685486 (+ 76us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.685660 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12744,"thread_start_us":12417,"threads_started":1}
I20260926 21:28:19.690544 2752 tablet_service.cc:1467] Processing CreateTablet for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc (DEFAULT_TABLE table=test-table [id=9950cfb1aa764935b807687ca27a5a98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 21:28:19.692387 2752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:19.696184 2836 tablet_bootstrap.cc:492] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Bootstrap starting.
I20260926 21:28:19.698453 2837 tablet_bootstrap.cc:492] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Bootstrap starting.
I20260926 21:28:19.701860 2836 tablet_bootstrap.cc:654] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:19.702116 2837 tablet_bootstrap.cc:654] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:19.703238 2837 log.cc:824] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:19.703243 2836 log.cc:824] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:19.706262 2837 tablet_bootstrap.cc:492] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: No bootstrap required, opened a new log
I20260926 21:28:19.706593 2837 ts_tablet_manager.cc:1397] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260926 21:28:19.706718 2836 tablet_bootstrap.cc:492] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: No bootstrap required, opened a new log
I20260926 21:28:19.707018 2836 ts_tablet_manager.cc:1397] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260926 21:28:19.708654 2840 tablet_bootstrap.cc:492] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Bootstrap starting.
I20260926 21:28:19.712494 2840 tablet_bootstrap.cc:654] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:19.713956 2840 log.cc:824] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:19.716797 2836 raft_consensus.cc:348] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.717226 2836 raft_consensus.cc:374] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:19.717330 2836 raft_consensus.cc:729] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7303d05b71cb4030a193516d4bf58f1a, State: Initialized, Role: FOLLOWER
I20260926 21:28:19.717418 2837 raft_consensus.cc:348] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.717833 2837 raft_consensus.cc:374] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:19.717942 2837 raft_consensus.cc:729] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd6996953980436b9c658f00bf6b1ecb, State: Initialized, Role: FOLLOWER
I20260926 21:28:19.717981 2840 tablet_bootstrap.cc:492] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: No bootstrap required, opened a new log
I20260926 21:28:19.718257 2840 ts_tablet_manager.cc:1397] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 21:28:19.718099 2836 consensus_queue.cc:260] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.718534 2837 consensus_queue.cc:260] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.720217 2836 ts_tablet_manager.cc:1428] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260926 21:28:19.728968 2840 raft_consensus.cc:348] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.729408 2840 raft_consensus.cc:374] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:19.729535 2840 raft_consensus.cc:729] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b5ee9bf4154b3c98c13c02cead1e39, State: Initialized, Role: FOLLOWER
I20260926 21:28:19.730183 2686 heartbeater.cc:503] Master 127.14.142.62:38187 was elected leader, sending a full tablet report...
I20260926 21:28:19.730232 2840 consensus_queue.cc:260] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.730901 2837 ts_tablet_manager.cc:1428] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Time spent starting tablet: real 0.024s user 0.020s sys 0.002s
I20260926 21:28:19.731822 2817 heartbeater.cc:503] Master 127.14.142.62:38187 was elected leader, sending a full tablet report...
I20260926 21:28:19.732426 2840 ts_tablet_manager.cc:1428] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260926 21:28:19.811895 2555 tablet.cc:2369] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:19.819725 2842 raft_consensus.cc:482] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 21:28:19.820214 2842 raft_consensus.cc:504] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.823520 2842 leader_election.cc:288] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd6996953980436b9c658f00bf6b1ecb (127.14.142.2:33159), 74b5ee9bf4154b3c98c13c02cead1e39 (127.14.142.3:45639)
W20260926 21:28:19.828135 2687 tablet.cc:2369] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:19.832077 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.824661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:57475 (local address 127.14.142.3:45639)
0926 21:28:19.824958 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.824968 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.825777 (+ 809us) server_negotiation.cc:407] Connection header received
0926 21:28:19.826046 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.826054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.826168 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.826283 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:19.826907 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.827679 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.829907 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.830774 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.830982 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.831079 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.831524 (+ 445us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.831769 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260926 21:28:19.832161 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.824103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.14.142.1:47489 (local address 127.14.142.2:33159)
0926 21:28:19.824457 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.824468 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.825244 (+ 776us) server_negotiation.cc:407] Connection header received
0926 21:28:19.825526 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.825532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.825622 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.825738 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 21:28:19.826405 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.827160 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.830310 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.831143 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.831360 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.831442 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.831764 (+ 322us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.831901 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260926 21:28:19.832186 2845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.824179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.3:45639 (local address 127.14.142.1:57475)
0926 21:28:19.825636 (+ 1457us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.825653 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.825881 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.826300 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.826305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.826330 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:19.826731 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.826741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.827859 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.827866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.829690 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.829706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.830982 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.830989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.831167 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.831194 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.831467 (+ 273us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.831709 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"mutex_wait_us":31,"thread_start_us":1010,"threads_started":1}
I20260926 21:28:19.832824 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.824104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.2:33159 (local address 127.14.142.1:47489)
0926 21:28:19.825104 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:19.825121 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:28:19.825352 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:28:19.825759 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:28:19.825764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:28:19.825794 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 21:28:19.826234 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.826247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.827316 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.827322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.830045 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:28:19.830067 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.831401 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.831410 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:28:19.831584 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.831613 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 21:28:19.831734 (+ 121us) client_negotiation.cc:240] Negotiation successful
0926 21:28:19.831876 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":516,"thread_start_us":613,"threads_started":1}
I20260926 21:28:19.833647 2772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291f3612eb5945d0bfe8fbbd9a5a0efc" candidate_uuid: "7303d05b71cb4030a193516d4bf58f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" is_pre_election: true
I20260926 21:28:19.833810 2640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291f3612eb5945d0bfe8fbbd9a5a0efc" candidate_uuid: "7303d05b71cb4030a193516d4bf58f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd6996953980436b9c658f00bf6b1ecb" is_pre_election: true
I20260926 21:28:19.834343 2772 raft_consensus.cc:2454] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7303d05b71cb4030a193516d4bf58f1a in term 0.
I20260926 21:28:19.834343 2640 raft_consensus.cc:2454] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7303d05b71cb4030a193516d4bf58f1a in term 0.
I20260926 21:28:19.835207 2442 leader_election.cc:302] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7303d05b71cb4030a193516d4bf58f1a, 74b5ee9bf4154b3c98c13c02cead1e39, fd6996953980436b9c658f00bf6b1ecb; no voters:
I20260926 21:28:19.835794 2842 raft_consensus.cc:2781] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 21:28:19.835989 2842 raft_consensus.cc:482] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 21:28:19.836149 2842 raft_consensus.cc:3037] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 0 FOLLOWER]: Advancing to term 1
W20260926 21:28:19.836251 2818 tablet.cc:2369] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:28:19.838785 2842 raft_consensus.cc:504] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.839660 2842 leader_election.cc:288] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [CANDIDATE]: Term 1 election: Requested vote from peers fd6996953980436b9c658f00bf6b1ecb (127.14.142.2:33159), 74b5ee9bf4154b3c98c13c02cead1e39 (127.14.142.3:45639)
I20260926 21:28:19.840517 2640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291f3612eb5945d0bfe8fbbd9a5a0efc" candidate_uuid: "7303d05b71cb4030a193516d4bf58f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd6996953980436b9c658f00bf6b1ecb"
I20260926 21:28:19.840723 2772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "291f3612eb5945d0bfe8fbbd9a5a0efc" candidate_uuid: "7303d05b71cb4030a193516d4bf58f1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b5ee9bf4154b3c98c13c02cead1e39"
I20260926 21:28:19.840863 2640 raft_consensus.cc:3037] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:19.841005 2772 raft_consensus.cc:3037] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:19.843485 2772 raft_consensus.cc:2454] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7303d05b71cb4030a193516d4bf58f1a in term 1.
I20260926 21:28:19.843621 2640 raft_consensus.cc:2454] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7303d05b71cb4030a193516d4bf58f1a in term 1.
I20260926 21:28:19.844153 2442 leader_election.cc:302] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7303d05b71cb4030a193516d4bf58f1a, 74b5ee9bf4154b3c98c13c02cead1e39; no voters:
I20260926 21:28:19.844621 2842 raft_consensus.cc:2781] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:19.845163 2842 raft_consensus.cc:686] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 1 LEADER]: Becoming Leader. State: Replica: 7303d05b71cb4030a193516d4bf58f1a, State: Running, Role: LEADER
I20260926 21:28:19.845728 2842 consensus_queue.cc:237] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } }
I20260926 21:28:19.851200 2367 catalog_manager.cc:5495] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7303d05b71cb4030a193516d4bf58f1a (127.14.142.1). New cstate: current_term: 1 leader_uuid: "7303d05b71cb4030a193516d4bf58f1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7303d05b71cb4030a193516d4bf58f1a" member_type: VOTER last_known_addr { host: "127.14.142.1" port: 36787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 } health_report { overall_health: UNKNOWN } } }
I20260926 21:28:19.913795 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.907562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53082 (local address 127.14.142.1:36787)
0926 21:28:19.907830 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.907835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.907852 (+ 17us) server_negotiation.cc:407] Connection header received
0926 21:28:19.907950 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.907953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.908259 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.908493 (+ 234us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:28:19.908944 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.909730 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.911460 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.911934 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.912036 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.912618 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:28:19.913219 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:28:19.913353 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.913485 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.913578 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260926 21:28:19.923353 2772 raft_consensus.cc:1261] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 1 FOLLOWER]: Refusing update from remote peer 7303d05b71cb4030a193516d4bf58f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:19.923329 2640 raft_consensus.cc:1261] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 1 FOLLOWER]: Refusing update from remote peer 7303d05b71cb4030a193516d4bf58f1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:28:19.924438 2847 consensus_queue.cc:1035] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd6996953980436b9c658f00bf6b1ecb" member_type: VOTER last_known_addr { host: "127.14.142.2" port: 33159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:19.924940 2842 consensus_queue.cc:1035] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b5ee9bf4154b3c98c13c02cead1e39" member_type: VOTER last_known_addr { host: "127.14.142.3" port: 45639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:28:19.940819 2851 mvcc.cc:205] Tried to move back new op lower bound from 7333716377275490304 to 7333716376972840960. Current Snapshot: MvccSnapshot[applied={T|T < 7333716377275490304}]
I20260926 21:28:19.970568 2368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:19.971073 2368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 21:28:19.976625 2368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 097b05ff4f3145d5bef988d7f3265eaa: Sending DeleteTablet for 3 replicas of tablet 291f3612eb5945d0bfe8fbbd9a5a0efc
I20260926 21:28:19.977910 2489 tablet_service.cc:1514] Processing DeleteTablet for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:19 UTC) from {username='slave'} at 127.0.0.1:53078
I20260926 21:28:19.978032 2620 tablet_service.cc:1514] Processing DeleteTablet for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:19 UTC) from {username='slave'} at 127.0.0.1:40094
I20260926 21:28:19.978631 2752 tablet_service.cc:1514] Processing DeleteTablet for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 21:28:19 UTC) from {username='slave'} at 127.0.0.1:56400
I20260926 21:28:19.978940 2860 tablet_replica.cc:330] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: stopping tablet replica
I20260926 21:28:19.979605 2860 raft_consensus.cc:2229] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:19.980177 2860 raft_consensus.cc:2258] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:19.981001 2861 tablet_replica.cc:330] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: stopping tablet replica
I20260926 21:28:19.981305 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2427
I20260926 21:28:19.981438 2861 raft_consensus.cc:2229] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 1 LEADER]: Raft consensus shutting down.
I20260926 21:28:19.982105 2861 raft_consensus.cc:2258] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:19.983340 2860 ts_tablet_manager.cc:1905] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:19.987913 2862 tablet_replica.cc:330] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: stopping tablet replica
I20260926 21:28:19.988379 2862 raft_consensus.cc:2229] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 21:28:19.989019 2862 raft_consensus.cc:2258] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 21:28:19.989140 2861 ts_tablet_manager.cc:1905] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:19.991453 2860 ts_tablet_manager.cc:1918] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:19.991701 2860 log.cc:1196] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-1/wal/wals/291f3612eb5945d0bfe8fbbd9a5a0efc
I20260926 21:28:19.992332 2860 ts_tablet_manager.cc:1939] T 291f3612eb5945d0bfe8fbbd9a5a0efc P fd6996953980436b9c658f00bf6b1ecb: Deleting consensus metadata
I20260926 21:28:19.994513 2354 catalog_manager.cc:4841] TS fd6996953980436b9c658f00bf6b1ecb (127.14.142.2:33159): tablet 291f3612eb5945d0bfe8fbbd9a5a0efc (table test-table [id=9950cfb1aa764935b807687ca27a5a98]) successfully deleted
I20260926 21:28:19.994637 2862 ts_tablet_manager.cc:1905] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 21:28:19.995272 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:19.984023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53098 (local address 127.14.142.1:36787)
0926 21:28:19.984235 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:19.984241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:19.984262 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:28:19.984323 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:19.984328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:19.984420 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:19.984612 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:19.985164 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.986746 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.988679 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:19.989329 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:19.989440 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:19.991402 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:19.991415 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:19.991432 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:19.992238 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:19.992321 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:19.992356 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:19.992443 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:19.994765 (+ 2322us) server_negotiation.cc:299] Negotiation successful
0926 21:28:19.994890 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 21:28:19.996399 2549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:20.002645 2862 ts_tablet_manager.cc:1918] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:20.002852 2862 log.cc:1196] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-2/wal/wals/291f3612eb5945d0bfe8fbbd9a5a0efc
I20260926 21:28:20.003435 2862 ts_tablet_manager.cc:1939] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 74b5ee9bf4154b3c98c13c02cead1e39: Deleting consensus metadata
I20260926 21:28:20.005827 2352 catalog_manager.cc:4841] TS 74b5ee9bf4154b3c98c13c02cead1e39 (127.14.142.3:45639): tablet 291f3612eb5945d0bfe8fbbd9a5a0efc (table test-table [id=9950cfb1aa764935b807687ca27a5a98]) successfully deleted
I20260926 21:28:20.121158 2861 ts_tablet_manager.cc:1918] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 21:28:20.121433 2861 log.cc:1196] T 291f3612eb5945d0bfe8fbbd9a5a0efc P 7303d05b71cb4030a193516d4bf58f1a: Deleting WAL directory at /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790457761955429-14904-0/minicluster-data/ts-0/wal/wals/291f3612eb5945d0bfe8fbbd9a5a0efc
I20260926 21:28:20.121941 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2427
I20260926 21:28:20.142793 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2558
W20260926 21:28:20.142686 2352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.14.142.1:36787 (error 108)
I20260926 21:28:20.144590 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.144034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53114)
0926 21:28:20.144233 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.144365 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260926 21:28:20.144945 2352 catalog_manager.cc:4584] TS 7303d05b71cb4030a193516d4bf58f1a (127.14.142.1:36787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 291f3612eb5945d0bfe8fbbd9a5a0efc: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
I20260926 21:28:20.150794 2833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.143637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40102 (local address 127.14.142.2:33159)
0926 21:28:20.143846 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:20.143854 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:20.143899 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:28:20.144097 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:20.144103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:20.144206 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:20.144310 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:20.144836 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:20.145708 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:20.146991 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:20.147425 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:20.147572 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:20.149063 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:20.149077 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:20.149198 (+ 121us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:20.149885 (+ 687us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:20.149938 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:20.149962 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:20.150135 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:20.150315 (+ 180us) server_negotiation.cc:299] Negotiation successful
0926 21:28:20.150446 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 21:28:20.151558 2681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:20.198607 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.198042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53130)
0926 21:28:20.198304 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.198413 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260926 21:28:20.199659 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.199296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53144)
0926 21:28:20.199461 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.199541 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 21:28:20.249243 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2558
I20260926 21:28:20.260085 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.259608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53152)
0926 21:28:20.259836 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.259936 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260926 21:28:20.261345 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.260859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53154)
0926 21:28:20.261122 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.261200 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260926 21:28:20.267302 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2690
I20260926 21:28:20.275921 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.268298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.14.142.3:45639)
0926 21:28:20.268515 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:20.268521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:20.268544 (+ 23us) server_negotiation.cc:407] Connection header received
0926 21:28:20.268740 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:20.268744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:20.268836 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:20.268952 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:20.269485 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:20.270486 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:20.271945 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:20.272444 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:20.272672 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:20.274296 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:20.274305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:20.274321 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:20.274960 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:20.275013 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:20.275035 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:20.275147 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:20.275301 (+ 154us) server_negotiation.cc:299] Negotiation successful
0926 21:28:20.275408 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 21:28:20.276983 2812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:20.341568 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.341121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53162)
0926 21:28:20.341316 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.341416 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 21:28:20.342960 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:20.342364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.1:36787 (local address 127.0.0.1:53170)
0926 21:28:20.342673 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 21:28:20.342775 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.1:36787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 21:28:20.371605 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2690
I20260926 21:28:20.391137 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2337
I20260926 21:28:20.392478 2398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:20.505882 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2337
2026-09-26T21:28:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3135 ms)
[----------] 6 tests from EncryptionPolicyTest (24001 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:20 dist-test-slave-4d5t krb5kdc[2871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:20 dist-test-slave-4d5t krb5kdc[2871](info): set up 2 sockets
Sep 26 21:28:20 dist-test-slave-4d5t krb5kdc[2871](info): commencing operation
krb5kdc: starting...
W20260926 21:28:21.649601 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:21 dist-test-slave-4d5t krb5kdc[2871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:21.946862 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:22.305932 2884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:22.306236 2884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:22.306409 2884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:22.315294 2884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:22.315425 2884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:22.315536 2884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:22.315624 2884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:22.315666 2884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:22.326221 2884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:22.327597 2884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:22.329020 2884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:22.341550 2892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:22.341663 2889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:22.341657 2890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:22.342842 2884 server_base.cc:1034] running on GCE node
I20260926 21:28:22.344197 2884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:22.346334 2884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:22.347594 2884 hybrid_clock.cc:648] HybridClock initialized: now 1790458102347498 us; error 61 us; skew 500 ppm
I20260926 21:28:22.348091 2884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:22 dist-test-slave-4d5t krb5kdc[2871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458102, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:22.359091 2884 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:22.361052 2884 webserver.cc:460] Webserver started at http://127.0.0.1:41719/ using document root <none> and password file <none>
I20260926 21:28:22.361826 2884 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:22.361987 2884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:22.362313 2884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:22.365690 2884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "89a3c0303fa94666bd45dcd4e8dc66fe"
format_stamp: "Formatted at 2026-09-26 21:28:22 on dist-test-slave-4d5t"
server_key: "f9b698c728342889362e1a867d9f51db"
server_key_iv: "ee30fe84ef3180d54f6c4bb1c1f5cc77"
server_key_version: "encryptionkey@0"
I20260926 21:28:22.366569 2884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "89a3c0303fa94666bd45dcd4e8dc66fe"
format_stamp: "Formatted at 2026-09-26 21:28:22 on dist-test-slave-4d5t"
server_key: "f9b698c728342889362e1a867d9f51db"
server_key_iv: "ee30fe84ef3180d54f6c4bb1c1f5cc77"
server_key_version: "encryptionkey@0"
I20260926 21:28:22.372759 2884 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260926 21:28:22.377137 2899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:22.378729 2884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 21:28:22.378947 2884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "89a3c0303fa94666bd45dcd4e8dc66fe"
format_stamp: "Formatted at 2026-09-26 21:28:22 on dist-test-slave-4d5t"
server_key: "f9b698c728342889362e1a867d9f51db"
server_key_iv: "ee30fe84ef3180d54f6c4bb1c1f5cc77"
server_key_version: "encryptionkey@0"
I20260926 21:28:22.379165 2884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:22.398816 2884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:22.403507 2884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:22.403852 2884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:22.425369 2884 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46147
I20260926 21:28:22.425402 2950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46147 every 8 connection(s)
I20260926 21:28:22.426869 2884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:22.427486 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2884
I20260926 21:28:22.427754 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:22.428671 14904 external_mini_cluster.cc:1428] Setting key d39cb2ed021e02a31c0430ac57b57bf1
I20260926 21:28:22.431649 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:22.442760 2951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe: Bootstrap starting.
Sep 26 21:28:22 dist-test-slave-4d5t krb5kdc[2871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 21:28:22.447216 2951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:22.448876 2951 log.cc:824] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:22.452811 2951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe: No bootstrap required, opened a new log
I20260926 21:28:22.455577 2954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:22.430651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50903 (local address 127.0.0.1:46147)
0926 21:28:22.431437 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:22.431452 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:22.431494 (+ 42us) server_negotiation.cc:407] Connection header received
0926 21:28:22.433395 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:22.433417 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:22.433937 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:22.434388 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:22.435998 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:22.437552 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:22.438961 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:22.439722 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:22.439982 (+ 260us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:22.444867 (+ 4885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:22.444886 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:22.444912 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:22.444997 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:22.449413 (+ 4416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:22.450713 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:22.450723 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:22.450732 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:22.450973 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:22.451807 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:22.451813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:22.451816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:22.452600 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:22.452961 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:22.453644 (+ 683us) server_negotiation.cc:299] Negotiation successful
0926 21:28:22.454051 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260926 21:28:22.460337 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:22.463100 2951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46147 } }
I20260926 21:28:22.463624 2951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:22.463815 2951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a3c0303fa94666bd45dcd4e8dc66fe, State: Initialized, Role: FOLLOWER
I20260926 21:28:22.464669 2951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46147 } }
I20260926 21:28:22.464958 2951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:22.465137 2951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:22.465366 2951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:22.468335 2951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46147 } }
I20260926 21:28:22.469012 2951 leader_election.cc:302] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89a3c0303fa94666bd45dcd4e8dc66fe; no voters:
I20260926 21:28:22.469848 2951 leader_election.cc:288] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:22.469980 2961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:22.471374 2961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [term 1 LEADER]: Becoming Leader. State: Replica: 89a3c0303fa94666bd45dcd4e8dc66fe, State: Running, Role: LEADER
I20260926 21:28:22.472133 2961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46147 } }
I20260926 21:28:22.472779 2951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:22.476082 2962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46147 } } }
I20260926 21:28:22.476809 2962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:22.479060 2966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:22.479344 2963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89a3c0303fa94666bd45dcd4e8dc66fe. Latest consensus state: current_term: 1 leader_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89a3c0303fa94666bd45dcd4e8dc66fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46147 } } }
I20260926 21:28:22.479875 2963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:22.491062 2966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:22.493832 2954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:22.465735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58365 (local address 127.0.0.1:46147)
0926 21:28:22.465989 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:22.466000 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:22.466553 (+ 553us) server_negotiation.cc:407] Connection header received
0926 21:28:22.466862 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:22.466867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:22.466957 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:22.467116 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:22.468983 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:22.470144 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:22.472355 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:22.473124 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:22.473289 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:22.478878 (+ 5589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:22.478887 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:22.478892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:22.478941 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:22.484163 (+ 5222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:22.485330 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:22.485340 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:22.485346 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:22.488297 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:22.489340 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:22.489353 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:22.489359 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:22.489989 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:22.492143 (+ 2154us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:22.492414 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 21:28:22.492723 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:28:22.503607 2966 catalog_manager.cc:1347] Generated new cluster ID: 06ba71dc90e644a0a1948478ac58e0c4
I20260926 21:28:22.503755 2966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:22.514770 2966 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:22.516161 2966 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:22.530112 2966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89a3c0303fa94666bd45dcd4e8dc66fe: Generated new TSK 0
I20260926 21:28:22.531138 2966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:22.547186 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2884
I20260926 21:28:22.548959 2946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:22.648394 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2884
W20260926 21:28:22.670565 2869 connection.cc:535] client connection to 127.0.0.1:46147 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46147): Connection reset by peer (error 104)
2026-09-26T21:28:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2134 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:22 dist-test-slave-4d5t krb5kdc[2983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:22 dist-test-slave-4d5t krb5kdc[2983](info): set up 2 sockets
Sep 26 21:28:22 dist-test-slave-4d5t krb5kdc[2983](info): commencing operation
krb5kdc: starting...
W20260926 21:28:23.797725 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.005s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:23 dist-test-slave-4d5t krb5kdc[2983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:24.096835 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:24.461232 2996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:24.461658 2996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:24.461828 2996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:24.469909 2996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:24.470080 2996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:24.470188 2996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:24.470302 2996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:24.470387 2996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:24.481171 2996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:24.482625 2996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:24.484005 2996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:24.497164 3001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:24.497182 3004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:24.497429 3002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:24.503600 2996 server_base.cc:1034] running on GCE node
I20260926 21:28:24.504616 2996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:24.506188 2996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:24.507494 2996 hybrid_clock.cc:648] HybridClock initialized: now 1790458104507443 us; error 48 us; skew 500 ppm
I20260926 21:28:24.507910 2996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:24 dist-test-slave-4d5t krb5kdc[2983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458104, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:24.511520 2996 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:24.513496 2996 webserver.cc:460] Webserver started at http://127.0.0.1:35877/ using document root <none> and password file <none>
I20260926 21:28:24.514266 2996 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:24.514436 2996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:24.514788 2996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:24.518083 2996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "b1bd006bb8a145ec80e45b1745addecb"
format_stamp: "Formatted at 2026-09-26 21:28:24 on dist-test-slave-4d5t"
server_key: "80a3b93add694bfdf2f72bd0918d8c56"
server_key_iv: "03846215f4a7eea38b6bda57a099c1a6"
server_key_version: "encryptionkey@0"
I20260926 21:28:24.518930 2996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "b1bd006bb8a145ec80e45b1745addecb"
format_stamp: "Formatted at 2026-09-26 21:28:24 on dist-test-slave-4d5t"
server_key: "80a3b93add694bfdf2f72bd0918d8c56"
server_key_iv: "03846215f4a7eea38b6bda57a099c1a6"
server_key_version: "encryptionkey@0"
I20260926 21:28:24.524608 2996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 21:28:24.528255 3011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:24.529811 2996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:28:24.530031 2996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "b1bd006bb8a145ec80e45b1745addecb"
format_stamp: "Formatted at 2026-09-26 21:28:24 on dist-test-slave-4d5t"
server_key: "80a3b93add694bfdf2f72bd0918d8c56"
server_key_iv: "03846215f4a7eea38b6bda57a099c1a6"
server_key_version: "encryptionkey@0"
I20260926 21:28:24.530258 2996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:24.555958 2996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:24.560988 2996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:24.561340 2996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:24.583335 2996 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36543
I20260926 21:28:24.583365 3062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36543 every 8 connection(s)
I20260926 21:28:24.584924 2996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:24.587219 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 2996
I20260926 21:28:24.587427 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:24.588001 14904 external_mini_cluster.cc:1428] Setting key aa899310f74361d7d8dd01fabba7a67c
I20260926 21:28:24.589463 3063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:24.602069 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:24.589915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48587 (local address 127.0.0.1:36543)
0926 21:28:24.590699 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:24.590711 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:24.590751 (+ 40us) server_negotiation.cc:407] Connection header received
0926 21:28:24.592346 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:24.592365 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:24.592816 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:24.593155 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:24.593795 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:24.595144 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:24.597206 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:24.597926 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:24.598084 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:24.599464 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:24.599472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:24.599489 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:24.600078 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:24.600161 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:24.600192 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:24.600320 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:24.600844 (+ 524us) server_negotiation.cc:299] Negotiation successful
0926 21:28:24.601035 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":360,"threads_started":1}
I20260926 21:28:24.603339 3063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb: Bootstrap starting.
I20260926 21:28:24.607039 3063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:24.607213 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:24.608124 3063 log.cc:824] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:24.610914 3063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb: No bootstrap required, opened a new log
I20260926 21:28:24.619642 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:24.612058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37537 (local address 127.0.0.1:36543)
0926 21:28:24.612268 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:24.612275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:24.612614 (+ 339us) server_negotiation.cc:407] Connection header received
0926 21:28:24.612846 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:24.612851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:24.612923 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:24.613020 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:24.613616 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:24.614476 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:24.616457 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:24.617049 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:24.617169 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:24.618420 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:24.618425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:24.618428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:24.618446 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:24.618484 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:24.618500 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:24.618595 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:24.619068 (+ 473us) server_negotiation.cc:299] Negotiation successful
0926 21:28:24.619200 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 21:28:24.621064 3063 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bd006bb8a145ec80e45b1745addecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260926 21:28:24.621551 3063 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:24.621670 3063 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1bd006bb8a145ec80e45b1745addecb, State: Initialized, Role: FOLLOWER
I20260926 21:28:24.622274 3063 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bd006bb8a145ec80e45b1745addecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260926 21:28:24.622478 3063 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:24.622586 3063 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:24.622711 3063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:24.625365 3063 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bd006bb8a145ec80e45b1745addecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260926 21:28:24.625826 3063 leader_election.cc:302] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1bd006bb8a145ec80e45b1745addecb; no voters:
I20260926 21:28:24.626389 3063 leader_election.cc:288] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:24.626569 3073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:24.627281 3073 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [term 1 LEADER]: Becoming Leader. State: Replica: b1bd006bb8a145ec80e45b1745addecb, State: Running, Role: LEADER
I20260926 21:28:24.627921 3073 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bd006bb8a145ec80e45b1745addecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } }
I20260926 21:28:24.628785 3063 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:24.630965 3075 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1bd006bb8a145ec80e45b1745addecb. Latest consensus state: current_term: 1 leader_uuid: "b1bd006bb8a145ec80e45b1745addecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bd006bb8a145ec80e45b1745addecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } } }
I20260926 21:28:24.630860 3074 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1bd006bb8a145ec80e45b1745addecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1bd006bb8a145ec80e45b1745addecb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36543 } } }
I20260926 21:28:24.631430 3074 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:24.631430 3075 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:24.633659 3079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:24.638818 3079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:24.649878 3079 catalog_manager.cc:1347] Generated new cluster ID: 1274146f7cb84b8ab44dc69fa950d2f1
I20260926 21:28:24.650012 3079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:24.663393 3079 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:24.665410 3079 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:24.678046 3079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1bd006bb8a145ec80e45b1745addecb: Generated new TSK 0
I20260926 21:28:24.679230 3079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:24.697918 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2996
W20260926 21:28:24.699409 3058 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48587
W20260926 21:28:24.700299 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 2996: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:24.700508 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 2996
W20260926 21:28:24.718519 2981 connection.cc:535] client connection to 127.0.0.1:36543 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36543): Connection reset by peer (error 104)
2026-09-26T21:28:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2048 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:24 dist-test-slave-4d5t krb5kdc[3094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:24 dist-test-slave-4d5t krb5kdc[3094](info): set up 2 sockets
Sep 26 21:28:24 dist-test-slave-4d5t krb5kdc[3094](info): commencing operation
krb5kdc: starting...
W20260926 21:28:25.893649 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:26 dist-test-slave-4d5t krb5kdc[3094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:26.192530 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:26.551889 3107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:26.552280 3107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:26.552457 3107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:26.560940 3107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:26.561134 3107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:26.561231 3107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:26.561363 3107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:26.561444 3107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:26.572077 3107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40217
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:26.573580 3107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:26.574991 3107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:26.587702 3113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:26.587705 3112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:26.587872 3115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:26.588634 3107 server_base.cc:1034] running on GCE node
I20260926 21:28:26.589468 3107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:26.590955 3107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:26.592190 3107 hybrid_clock.cc:648] HybridClock initialized: now 1790458106592151 us; error 28 us; skew 500 ppm
I20260926 21:28:26.592507 3107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:26 dist-test-slave-4d5t krb5kdc[3094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458106, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:26.596307 3107 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:26.598150 3107 webserver.cc:460] Webserver started at http://127.0.0.1:44387/ using document root <none> and password file <none>
I20260926 21:28:26.598861 3107 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:26.598966 3107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:26.599225 3107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:26.602360 3107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "56863e82fb6c4fbda47a72f939a6a244"
format_stamp: "Formatted at 2026-09-26 21:28:26 on dist-test-slave-4d5t"
server_key: "f43a3ad0b1b155b336255d684c63d81c"
server_key_iv: "acf9dfccbee8db924753c078b9e10a68"
server_key_version: "encryptionkey@0"
I20260926 21:28:26.603098 3107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "56863e82fb6c4fbda47a72f939a6a244"
format_stamp: "Formatted at 2026-09-26 21:28:26 on dist-test-slave-4d5t"
server_key: "f43a3ad0b1b155b336255d684c63d81c"
server_key_iv: "acf9dfccbee8db924753c078b9e10a68"
server_key_version: "encryptionkey@0"
I20260926 21:28:26.609129 3107 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 21:28:26.612903 3123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:26.614756 3107 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 21:28:26.614979 3107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "56863e82fb6c4fbda47a72f939a6a244"
format_stamp: "Formatted at 2026-09-26 21:28:26 on dist-test-slave-4d5t"
server_key: "f43a3ad0b1b155b336255d684c63d81c"
server_key_iv: "acf9dfccbee8db924753c078b9e10a68"
server_key_version: "encryptionkey@0"
I20260926 21:28:26.615214 3107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:26.621697 3107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:26.626376 3107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:26.626722 3107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:26.647192 3107 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40217
I20260926 21:28:26.647501 3174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40217 every 8 connection(s)
I20260926 21:28:26.648756 3107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:26.652659 3175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:26.653446 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3107
I20260926 21:28:26.653642 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:26.654085 14904 external_mini_cluster.cc:1428] Setting key de1010fa9b9b7f991c0f77426649f236
I20260926 21:28:26.665206 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:26.656263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34665 (local address 127.0.0.1:40217)
0926 21:28:26.657452 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:26.657475 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:26.657548 (+ 73us) server_negotiation.cc:407] Connection header received
0926 21:28:26.660912 (+ 3364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:26.660949 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:26.661500 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:26.662023 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:26.663124 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:26.663133 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:26.663151 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:26.663169 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:26.663244 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:26.663274 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:26.663506 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:26.663628 (+ 122us) server_negotiation.cc:299] Negotiation successful
0926 21:28:26.663851 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":501,"threads_started":1}
I20260926 21:28:26.668927 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244: Bootstrap starting.
I20260926 21:28:26.669117 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:26.673192 3175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:26.674993 3175 log.cc:824] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:26.679613 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244: No bootstrap required, opened a new log
I20260926 21:28:26.679922 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:26.677285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35949 (local address 127.0.0.1:40217)
0926 21:28:26.677533 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:26.677541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:26.677736 (+ 195us) server_negotiation.cc:407] Connection header received
0926 21:28:26.677901 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:26.677907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:26.677986 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:26.678301 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:26.679223 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:26.679229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:26.679234 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:26.679253 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:26.679299 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:26.679320 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:26.679453 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:26.679509 (+ 56us) server_negotiation.cc:299] Negotiation successful
0926 21:28:26.679572 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 21:28:26.687377 3175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56863e82fb6c4fbda47a72f939a6a244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260926 21:28:26.687783 3175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:26.687915 3175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56863e82fb6c4fbda47a72f939a6a244, State: Initialized, Role: FOLLOWER
I20260926 21:28:26.688601 3175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56863e82fb6c4fbda47a72f939a6a244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260926 21:28:26.688846 3175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:26.688998 3175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:26.689181 3175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:26.691504 3175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56863e82fb6c4fbda47a72f939a6a244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260926 21:28:26.692677 3175 leader_election.cc:302] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56863e82fb6c4fbda47a72f939a6a244; no voters:
I20260926 21:28:26.693373 3175 leader_election.cc:288] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:26.693497 3185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:26.694311 3185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [term 1 LEADER]: Becoming Leader. State: Replica: 56863e82fb6c4fbda47a72f939a6a244, State: Running, Role: LEADER
I20260926 21:28:26.694922 3185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56863e82fb6c4fbda47a72f939a6a244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } }
I20260926 21:28:26.696026 3175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:26.699134 3186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56863e82fb6c4fbda47a72f939a6a244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56863e82fb6c4fbda47a72f939a6a244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } } }
I20260926 21:28:26.699173 3187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56863e82fb6c4fbda47a72f939a6a244. Latest consensus state: current_term: 1 leader_uuid: "56863e82fb6c4fbda47a72f939a6a244" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56863e82fb6c4fbda47a72f939a6a244" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40217 } } }
I20260926 21:28:26.699682 3186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:26.699699 3187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:26.701083 3193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:26.716508 3193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:26.726850 3193 catalog_manager.cc:1347] Generated new cluster ID: 3bb62d092a904165879fffc2abc7974c
I20260926 21:28:26.726997 3193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:26.736902 3193 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:26.737668 3193 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:26.747440 3193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56863e82fb6c4fbda47a72f939a6a244: Generated new TSK 0
I20260926 21:28:26.748481 3193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:26.756942 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3107
W20260926 21:28:26.758289 3170 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34665
W20260926 21:28:26.759027 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3107: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:26.759186 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3107
2026-09-26T21:28:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2058 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:26 dist-test-slave-4d5t krb5kdc[3206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:26 dist-test-slave-4d5t krb5kdc[3206](info): set up 2 sockets
Sep 26 21:28:26 dist-test-slave-4d5t krb5kdc[3206](info): commencing operation
krb5kdc: starting...
W20260926 21:28:27.930124 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:28 dist-test-slave-4d5t krb5kdc[3206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:28.227801 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:28.590175 3219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:28.590487 3219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:28.590662 3219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:28.598680 3219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:28.598802 3219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:28.598850 3219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:28.598945 3219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:28.599027 3219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:28.609453 3219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:28.610798 3219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:28.612172 3219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:28.623937 3224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:28.624014 3225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:28.624014 3227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:28.624898 3219 server_base.cc:1034] running on GCE node
I20260926 21:28:28.625751 3219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:28.627208 3219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:28.628448 3219 hybrid_clock.cc:648] HybridClock initialized: now 1790458108628401 us; error 40 us; skew 500 ppm
I20260926 21:28:28.628835 3219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:28 dist-test-slave-4d5t krb5kdc[3206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458108, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:28.637904 3219 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:28.640130 3219 webserver.cc:460] Webserver started at http://127.0.0.1:36179/ using document root <none> and password file <none>
I20260926 21:28:28.641135 3219 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:28.641281 3219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:28.641603 3219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:28.645381 3219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "375bf1c13b754127baa7c841c15fef45"
format_stamp: "Formatted at 2026-09-26 21:28:28 on dist-test-slave-4d5t"
server_key: "21ddf19918a38c299505839d36a2c01e"
server_key_iv: "0091268ce7e0130753e4125e3d9b64e1"
server_key_version: "encryptionkey@0"
I20260926 21:28:28.646173 3219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "375bf1c13b754127baa7c841c15fef45"
format_stamp: "Formatted at 2026-09-26 21:28:28 on dist-test-slave-4d5t"
server_key: "21ddf19918a38c299505839d36a2c01e"
server_key_iv: "0091268ce7e0130753e4125e3d9b64e1"
server_key_version: "encryptionkey@0"
I20260926 21:28:28.651660 3219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260926 21:28:28.655491 3234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:28.657176 3219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:28:28.657384 3219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "375bf1c13b754127baa7c841c15fef45"
format_stamp: "Formatted at 2026-09-26 21:28:28 on dist-test-slave-4d5t"
server_key: "21ddf19918a38c299505839d36a2c01e"
server_key_iv: "0091268ce7e0130753e4125e3d9b64e1"
server_key_version: "encryptionkey@0"
I20260926 21:28:28.657625 3219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:28.669902 3219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:28.674496 3219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:28.674907 3219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:28.695520 3219 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34619
I20260926 21:28:28.695624 3285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34619 every 8 connection(s)
I20260926 21:28:28.697119 3219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:28.700999 3286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:28.707037 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3219
I20260926 21:28:28.707228 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:28.707748 14904 external_mini_cluster.cc:1428] Setting key 0bf7dbb33289a603bf2fa9b71c88ea34
I20260926 21:28:28.715701 3286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45: Bootstrap starting.
I20260926 21:28:28.720055 3286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:28.721433 3286 log.cc:824] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45: Log is configured to *not* fsync() on all Append() calls
Sep 26 21:28:28 dist-test-slave-4d5t krb5kdc[3206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458108, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 21:28:28.725037 3286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45: No bootstrap required, opened a new log
I20260926 21:28:28.733503 3286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375bf1c13b754127baa7c841c15fef45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34619 } }
I20260926 21:28:28.733952 3286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:28.734072 3286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375bf1c13b754127baa7c841c15fef45, State: Initialized, Role: FOLLOWER
I20260926 21:28:28.734840 3286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375bf1c13b754127baa7c841c15fef45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34619 } }
I20260926 21:28:28.735132 3286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:28.735291 3286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:28.735509 3286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:28.736241 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:28.709710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59331 (local address 127.0.0.1:34619)
0926 21:28:28.710650 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:28.710666 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:28.710711 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:28:28.712939 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:28.712968 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:28.713451 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:28.713885 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:28.716627 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:28.718043 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:28.719944 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:28.720910 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:28.721085 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:28.726251 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:28.726261 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:28.726278 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:28.726320 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:28.731030 (+ 4710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:28.732131 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:28.732141 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:28.732149 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:28.732388 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:28.732873 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:28.732879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:28.732884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:28.733505 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:28.733861 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:28.734370 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 21:28:28.734893 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":463,"threads_started":1}
I20260926 21:28:28.738621 3286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375bf1c13b754127baa7c841c15fef45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34619 } }
I20260926 21:28:28.739262 3286 leader_election.cc:302] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375bf1c13b754127baa7c841c15fef45; no voters:
I20260926 21:28:28.740013 3286 leader_election.cc:288] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:28.740181 3291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:28.740661 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:28.741544 3291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [term 1 LEADER]: Becoming Leader. State: Replica: 375bf1c13b754127baa7c841c15fef45, State: Running, Role: LEADER
I20260926 21:28:28.742148 3291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375bf1c13b754127baa7c841c15fef45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34619 } }
I20260926 21:28:28.742339 3286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:28.744894 3292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "375bf1c13b754127baa7c841c15fef45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375bf1c13b754127baa7c841c15fef45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34619 } } }
I20260926 21:28:28.745316 3292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:28.745774 3293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375bf1c13b754127baa7c841c15fef45. Latest consensus state: current_term: 1 leader_uuid: "375bf1c13b754127baa7c841c15fef45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375bf1c13b754127baa7c841c15fef45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34619 } } }
I20260926 21:28:28.746058 3293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:28.748896 3297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:28.754698 3297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:28.769429 3297 catalog_manager.cc:1347] Generated new cluster ID: 0da79854df384caa8b958c21997f6789
I20260926 21:28:28.769593 3297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:28.784502 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:28.764943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59591 (local address 127.0.0.1:34619)
0926 21:28:28.765172 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:28.765181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:28.767150 (+ 1969us) server_negotiation.cc:407] Connection header received
0926 21:28:28.767356 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:28.767362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:28.767444 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:28.767576 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:28.769797 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:28.770943 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:28.772804 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:28.773469 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:28.773566 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:28.776926 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:28.776932 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:28.776937 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:28.777000 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:28.780544 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:28.781456 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:28.781462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:28.781466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:28.781624 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:28.782049 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:28.782053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:28.782057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:28.782439 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:28.782672 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:28.783870 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0926 21:28:28.784125 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 21:28:28.791126 3297 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:28.792779 3297 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:28.801075 3297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375bf1c13b754127baa7c841c15fef45: Generated new TSK 0
I20260926 21:28:28.802101 3297 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:28.809492 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3219
I20260926 21:28:28.811311 3281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:28.903054 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3219
2026-09-26T21:28:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:29 dist-test-slave-4d5t krb5kdc[3318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:29 dist-test-slave-4d5t krb5kdc[3318](info): set up 2 sockets
Sep 26 21:28:29 dist-test-slave-4d5t krb5kdc[3318](info): commencing operation
krb5kdc: starting...
W20260926 21:28:30.048142 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:30 dist-test-slave-4d5t krb5kdc[3318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:30.359752 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:30.708231 3331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:30.708633 3331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:30.708745 3331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:30.716346 3331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:30.716521 3331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:30.716643 3331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:30.716717 3331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:30.716755 3331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:30.727334 3331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:30.728760 3331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:30.730042 3331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:30.742620 3336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:30.742604 3337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:30.743260 3339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:30.743717 3331 server_base.cc:1034] running on GCE node
I20260926 21:28:30.744798 3331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:30.746333 3331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:30.747599 3331 hybrid_clock.cc:648] HybridClock initialized: now 1790458110747565 us; error 62 us; skew 500 ppm
I20260926 21:28:30.748011 3331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:30 dist-test-slave-4d5t krb5kdc[3318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:30.752386 3331 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:30.754774 3331 webserver.cc:460] Webserver started at http://127.0.0.1:37427/ using document root <none> and password file <none>
I20260926 21:28:30.755781 3331 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:30.756038 3331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:30.756726 3331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:30.761206 3331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "55ac6ea11298410a9db65e9386a584c3"
format_stamp: "Formatted at 2026-09-26 21:28:30 on dist-test-slave-4d5t"
server_key: "9f71e5af5f54d4584b70cf5a1a9883b8"
server_key_iv: "94c2bddf79b890116d28a9c004a9ffd8"
server_key_version: "encryptionkey@0"
I20260926 21:28:30.762351 3331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "55ac6ea11298410a9db65e9386a584c3"
format_stamp: "Formatted at 2026-09-26 21:28:30 on dist-test-slave-4d5t"
server_key: "9f71e5af5f54d4584b70cf5a1a9883b8"
server_key_iv: "94c2bddf79b890116d28a9c004a9ffd8"
server_key_version: "encryptionkey@0"
I20260926 21:28:30.769629 3331 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260926 21:28:30.774125 3346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:30.775920 3331 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 21:28:30.776271 3331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "55ac6ea11298410a9db65e9386a584c3"
format_stamp: "Formatted at 2026-09-26 21:28:30 on dist-test-slave-4d5t"
server_key: "9f71e5af5f54d4584b70cf5a1a9883b8"
server_key_iv: "94c2bddf79b890116d28a9c004a9ffd8"
server_key_version: "encryptionkey@0"
I20260926 21:28:30.776710 3331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:30.806742 3331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:30.812281 3331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:30.812739 3331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:30.833797 3331 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42203
I20260926 21:28:30.833835 3397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42203 every 8 connection(s)
I20260926 21:28:30.835282 3331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:30.839303 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3331
I20260926 21:28:30.839507 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:30.839499 3398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:30.839967 14904 external_mini_cluster.cc:1428] Setting key b55bcf85757efe72615ae57030b2a992
I20260926 21:28:30.856902 3398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3: Bootstrap starting.
I20260926 21:28:30.857267 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:30.841765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37369 (local address 127.0.0.1:42203)
0926 21:28:30.842641 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:30.842663 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:30.842717 (+ 54us) server_negotiation.cc:407] Connection header received
0926 21:28:30.845256 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:30.845291 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:30.845922 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:30.846430 (+ 508us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:30.847022 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:30.849324 (+ 2302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:30.850909 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:30.852096 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:30.852306 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:30.853991 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:30.854003 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:30.854024 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:30.854875 (+ 851us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:30.854987 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:30.855022 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:30.855166 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:30.855484 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 21:28:30.855791 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":346,"threads_started":1}
I20260926 21:28:30.861737 3398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:30.862438 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:30.863307 3398 log.cc:824] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:30.867372 3398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3: No bootstrap required, opened a new log
I20260926 21:28:30.876619 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:30.868823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58437 (local address 127.0.0.1:42203)
0926 21:28:30.869039 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:30.869046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:30.869563 (+ 517us) server_negotiation.cc:407] Connection header received
0926 21:28:30.869741 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:30.869750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:30.869835 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:30.869965 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:30.870580 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:30.871746 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:30.873373 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:30.874114 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:30.874256 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:30.875694 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:30.875701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:30.875706 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:30.875736 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:30.875785 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:30.875811 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:30.875928 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:30.876107 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 21:28:30.876202 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:28:30.878371 3398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ac6ea11298410a9db65e9386a584c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42203 } }
I20260926 21:28:30.878799 3398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:30.878916 3398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55ac6ea11298410a9db65e9386a584c3, State: Initialized, Role: FOLLOWER
I20260926 21:28:30.879511 3398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ac6ea11298410a9db65e9386a584c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42203 } }
I20260926 21:28:30.879731 3398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:30.879900 3398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:30.880082 3398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:30.882932 3398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ac6ea11298410a9db65e9386a584c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42203 } }
I20260926 21:28:30.883527 3398 leader_election.cc:302] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55ac6ea11298410a9db65e9386a584c3; no voters:
I20260926 21:28:30.884140 3398 leader_election.cc:288] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:30.884320 3408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:30.885787 3408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [term 1 LEADER]: Becoming Leader. State: Replica: 55ac6ea11298410a9db65e9386a584c3, State: Running, Role: LEADER
I20260926 21:28:30.886303 3408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ac6ea11298410a9db65e9386a584c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42203 } }
I20260926 21:28:30.886495 3398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:30.889213 3409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55ac6ea11298410a9db65e9386a584c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ac6ea11298410a9db65e9386a584c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42203 } } }
I20260926 21:28:30.889241 3410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ac6ea11298410a9db65e9386a584c3. Latest consensus state: current_term: 1 leader_uuid: "55ac6ea11298410a9db65e9386a584c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ac6ea11298410a9db65e9386a584c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42203 } } }
I20260926 21:28:30.889693 3410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:30.889693 3409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:30.891260 3415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:30.898633 3415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:30.909538 3415 catalog_manager.cc:1347] Generated new cluster ID: 40d8798d13f049ae99e80c56ea18be60
I20260926 21:28:30.909696 3415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:30.938692 3415 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:30.940181 3415 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:30.951090 3415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55ac6ea11298410a9db65e9386a584c3: Generated new TSK 0
I20260926 21:28:30.952098 3415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:30.973538 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3331
W20260926 21:28:30.974808 3393 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37369
W20260926 21:28:30.975627 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3331: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:30.975782 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3331
2026-09-26T21:28:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2072 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:31 dist-test-slave-4d5t krb5kdc[3429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:31 dist-test-slave-4d5t krb5kdc[3429](info): set up 2 sockets
Sep 26 21:28:31 dist-test-slave-4d5t krb5kdc[3429](info): commencing operation
krb5kdc: starting...
W20260926 21:28:32.138095 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:32 dist-test-slave-4d5t krb5kdc[3429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458112, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:32.434230 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:32.808522 3442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:32.808964 3442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:32.809247 3442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:32.818384 3442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:32.818593 3442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:32.818707 3442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:32.818813 3442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:32.818889 3442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:32.829754 3442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:32.831251 3442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:32.833447 3442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:32.849231 3447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:32.849472 3448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:32.849560 3450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:32.851593 3442 server_base.cc:1034] running on GCE node
I20260926 21:28:32.852962 3442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:32.854996 3442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:32.856316 3442 hybrid_clock.cc:648] HybridClock initialized: now 1790458112856215 us; error 87 us; skew 500 ppm
I20260926 21:28:32.856889 3442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:32 dist-test-slave-4d5t krb5kdc[3429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458112, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:32.860771 3442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:32.862588 3442 webserver.cc:460] Webserver started at http://127.0.0.1:35043/ using document root <none> and password file <none>
I20260926 21:28:32.863394 3442 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:32.863565 3442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:32.863909 3442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:32.867413 3442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "5f5cabc1ee234f1289b4bbb8e0a29256"
format_stamp: "Formatted at 2026-09-26 21:28:32 on dist-test-slave-4d5t"
server_key: "f3c318210435004581647fc83de0a469"
server_key_iv: "26a7b6d2b5b721a746f82ae8cb2f62b1"
server_key_version: "encryptionkey@0"
I20260926 21:28:32.868268 3442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "5f5cabc1ee234f1289b4bbb8e0a29256"
format_stamp: "Formatted at 2026-09-26 21:28:32 on dist-test-slave-4d5t"
server_key: "f3c318210435004581647fc83de0a469"
server_key_iv: "26a7b6d2b5b721a746f82ae8cb2f62b1"
server_key_version: "encryptionkey@0"
I20260926 21:28:32.875892 3442 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260926 21:28:32.880532 3457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:32.882637 3442 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:28:32.882874 3442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "5f5cabc1ee234f1289b4bbb8e0a29256"
format_stamp: "Formatted at 2026-09-26 21:28:32 on dist-test-slave-4d5t"
server_key: "f3c318210435004581647fc83de0a469"
server_key_iv: "26a7b6d2b5b721a746f82ae8cb2f62b1"
server_key_version: "encryptionkey@0"
I20260926 21:28:32.883122 3442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:32.889588 3442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:32.894519 3442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:32.895061 3442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:32.916985 3442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38915
I20260926 21:28:32.917183 3508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38915 every 8 connection(s)
I20260926 21:28:32.918452 3442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:32.922883 3509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:32.924823 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3442
I20260926 21:28:32.925040 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:32.925655 14904 external_mini_cluster.cc:1428] Setting key d9e9320b2e1f2a6fab4e55e217ca8e43
I20260926 21:28:32.934624 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:32.927823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48239 (local address 127.0.0.1:38915)
0926 21:28:32.928859 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:32.928872 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:32.928970 (+ 98us) server_negotiation.cc:407] Connection header received
0926 21:28:32.930819 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:32.930845 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:32.931259 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:32.931737 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:32.932964 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:32.932971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:32.932985 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:32.933001 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:32.933068 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:32.933093 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:32.933335 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:32.933472 (+ 137us) server_negotiation.cc:299] Negotiation successful
0926 21:28:32.933670 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":544,"threads_started":1}
I20260926 21:28:32.939670 3509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256: Bootstrap starting.
I20260926 21:28:32.940227 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:32.943354 3509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:32.944531 3509 log.cc:824] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:32.947952 3509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256: No bootstrap required, opened a new log
I20260926 21:28:32.948357 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:32.945059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39229 (local address 127.0.0.1:38915)
0926 21:28:32.945253 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:32.945262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:32.945963 (+ 701us) server_negotiation.cc:407] Connection header received
0926 21:28:32.946201 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:32.946206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:32.946259 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:32.946381 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:32.947389 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:32.947396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:32.947401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:32.947429 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:32.947478 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:32.947496 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:32.947632 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:32.947732 (+ 100us) server_negotiation.cc:299] Negotiation successful
0926 21:28:32.947803 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 21:28:32.957464 3509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38915 } }
I20260926 21:28:32.957888 3509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:32.957986 3509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f5cabc1ee234f1289b4bbb8e0a29256, State: Initialized, Role: FOLLOWER
I20260926 21:28:32.958734 3509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38915 } }
I20260926 21:28:32.959048 3509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:32.959211 3509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:32.959419 3509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:32.962289 3509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38915 } }
I20260926 21:28:32.962824 3509 leader_election.cc:302] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f5cabc1ee234f1289b4bbb8e0a29256; no voters:
I20260926 21:28:32.963476 3509 leader_election.cc:288] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:32.963688 3519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:32.964478 3519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [term 1 LEADER]: Becoming Leader. State: Replica: 5f5cabc1ee234f1289b4bbb8e0a29256, State: Running, Role: LEADER
I20260926 21:28:32.965143 3519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38915 } }
I20260926 21:28:32.965788 3509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:32.968106 3520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38915 } } }
I20260926 21:28:32.968678 3521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f5cabc1ee234f1289b4bbb8e0a29256. Latest consensus state: current_term: 1 leader_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f5cabc1ee234f1289b4bbb8e0a29256" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38915 } } }
I20260926 21:28:32.969043 3520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:32.969087 3521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:32.971585 3524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:32.978219 3524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:32.989008 3524 catalog_manager.cc:1347] Generated new cluster ID: 4818c4ffe7a740cb867a1e292106b13b
I20260926 21:28:32.989346 3524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:33.008615 3524 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:33.009265 3524 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:33.016073 3524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f5cabc1ee234f1289b4bbb8e0a29256: Generated new TSK 0
I20260926 21:28:33.017222 3524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:33.026702 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3442
W20260926 21:28:33.027832 3504 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48239
W20260926 21:28:33.028630 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3442: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:33.028782 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3442
2026-09-26T21:28:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2051 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:33 dist-test-slave-4d5t krb5kdc[3540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:33 dist-test-slave-4d5t krb5kdc[3540](info): set up 2 sockets
Sep 26 21:28:33 dist-test-slave-4d5t krb5kdc[3540](info): commencing operation
krb5kdc: starting...
W20260926 21:28:34.250655 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:34 dist-test-slave-4d5t krb5kdc[3540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:34.549808 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:34.918987 3553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:34.919363 3553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:34.919539 3553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:34.926942 3553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:34.927119 3553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:34.927229 3553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:34.927337 3553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:34.927428 3553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:34.937772 3553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:34.939203 3553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:34.940613 3553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:34.954161 3561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:34.954247 3559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:34.954180 3558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:34.955670 3553 server_base.cc:1034] running on GCE node
I20260926 21:28:34.956658 3553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:34.959129 3553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:34.960465 3553 hybrid_clock.cc:648] HybridClock initialized: now 1790458114960393 us; error 55 us; skew 500 ppm
I20260926 21:28:34.961023 3553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:34 dist-test-slave-4d5t krb5kdc[3540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:34.964537 3553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:34.966564 3553 webserver.cc:460] Webserver started at http://127.0.0.1:42081/ using document root <none> and password file <none>
I20260926 21:28:34.967350 3553 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:34.967518 3553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:34.967859 3553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:34.971493 3553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "5226feb0b179490c89fe3d62dac4ff96"
format_stamp: "Formatted at 2026-09-26 21:28:34 on dist-test-slave-4d5t"
server_key: "02271c267a281108a5a15e7da6187248"
server_key_iv: "d46db26130abe9a5587973f2ad4acc61"
server_key_version: "encryptionkey@0"
I20260926 21:28:34.972503 3553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "5226feb0b179490c89fe3d62dac4ff96"
format_stamp: "Formatted at 2026-09-26 21:28:34 on dist-test-slave-4d5t"
server_key: "02271c267a281108a5a15e7da6187248"
server_key_iv: "d46db26130abe9a5587973f2ad4acc61"
server_key_version: "encryptionkey@0"
I20260926 21:28:34.978745 3553 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260926 21:28:34.982873 3568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:34.985044 3553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:28:34.985313 3553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "5226feb0b179490c89fe3d62dac4ff96"
format_stamp: "Formatted at 2026-09-26 21:28:34 on dist-test-slave-4d5t"
server_key: "02271c267a281108a5a15e7da6187248"
server_key_iv: "d46db26130abe9a5587973f2ad4acc61"
server_key_version: "encryptionkey@0"
I20260926 21:28:34.985589 3553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:35.008585 3553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:35.013833 3553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:35.014173 3553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:35.036685 3553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38267
I20260926 21:28:35.036815 3619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38267 every 8 connection(s)
I20260926 21:28:35.038251 3553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:35.040975 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3553
I20260926 21:28:35.041158 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:35.041620 14904 external_mini_cluster.cc:1428] Setting key 280d360c50023b228f8b74578c325862
I20260926 21:28:35.042579 3620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:35.058779 3620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96: Bootstrap starting.
Sep 26 21:28:35 dist-test-slave-4d5t krb5kdc[3540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 21:28:35.063197 3620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:35.064432 3620 log.cc:824] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:35.068091 3620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96: No bootstrap required, opened a new log
I20260926 21:28:35.073401 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:35.043478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:53667 (local address 127.0.0.1:38267)
0926 21:28:35.044309 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:35.044322 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:35.044373 (+ 51us) server_negotiation.cc:407] Connection header received
0926 21:28:35.046086 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:35.046118 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:35.046576 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:35.047376 (+ 800us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:35.049369 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:35.052013 (+ 2644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:35.054827 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:35.056633 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:35.057095 (+ 462us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:35.063863 (+ 6768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:35.063877 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:35.063903 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:35.063979 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:35.068225 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:35.069435 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:35.069444 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:35.069452 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:35.069687 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:35.070248 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:35.070252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:35.070255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:35.070873 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:35.071157 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:35.071674 (+ 517us) server_negotiation.cc:299] Negotiation successful
0926 21:28:35.072084 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":354,"threads_started":1}
I20260926 21:28:35.077380 3620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226feb0b179490c89fe3d62dac4ff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260926 21:28:35.077740 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:35.077795 3620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:35.077899 3620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5226feb0b179490c89fe3d62dac4ff96, State: Initialized, Role: FOLLOWER
I20260926 21:28:35.078513 3620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226feb0b179490c89fe3d62dac4ff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260926 21:28:35.078789 3620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:35.078927 3620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:35.079092 3620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:35.082547 3620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226feb0b179490c89fe3d62dac4ff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260926 21:28:35.083199 3620 leader_election.cc:302] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5226feb0b179490c89fe3d62dac4ff96; no voters:
I20260926 21:28:35.083910 3620 leader_election.cc:288] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:35.084057 3630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:35.085271 3630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [term 1 LEADER]: Becoming Leader. State: Replica: 5226feb0b179490c89fe3d62dac4ff96, State: Running, Role: LEADER
I20260926 21:28:35.085935 3630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226feb0b179490c89fe3d62dac4ff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } }
I20260926 21:28:35.086282 3620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:35.092279 3631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5226feb0b179490c89fe3d62dac4ff96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226feb0b179490c89fe3d62dac4ff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260926 21:28:35.092840 3631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:35.092808 3632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5226feb0b179490c89fe3d62dac4ff96. Latest consensus state: current_term: 1 leader_uuid: "5226feb0b179490c89fe3d62dac4ff96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226feb0b179490c89fe3d62dac4ff96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38267 } } }
I20260926 21:28:35.093084 3632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:35.107368 3639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:35.107554 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:35.083009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:51643 (local address 127.0.0.1:38267)
0926 21:28:35.083244 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:35.083254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:35.083878 (+ 624us) server_negotiation.cc:407] Connection header received
0926 21:28:35.084135 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:35.084149 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:35.084244 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:35.084385 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:35.086720 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:35.088125 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:35.090813 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:35.091824 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:35.091990 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:35.096423 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:35.096433 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:35.096441 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:35.096499 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:35.101394 (+ 4895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:35.102966 (+ 1572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:35.102975 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:35.102982 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:35.103211 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:35.103920 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:35.103928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:35.103934 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:35.104609 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:35.105020 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:35.106738 (+ 1718us) server_negotiation.cc:299] Negotiation successful
0926 21:28:35.106973 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 21:28:35.113478 3639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:35.122291 3639 catalog_manager.cc:1347] Generated new cluster ID: 8932c2fefec04206a3fd507832adb653
I20260926 21:28:35.122450 3639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:35.152323 3639 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:35.154287 3639 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:35.171209 3639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5226feb0b179490c89fe3d62dac4ff96: Generated new TSK 0
I20260926 21:28:35.172531 3639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:35.183266 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3553
I20260926 21:28:35.184993 3615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:35.286206 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3553
W20260926 21:28:35.306821 3538 connection.cc:535] client connection to 127.0.0.1:38267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38267): Connection reset by peer (error 104)
2026-09-26T21:28:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2260 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:35 dist-test-slave-4d5t krb5kdc[3652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:35 dist-test-slave-4d5t krb5kdc[3652](info): set up 2 sockets
Sep 26 21:28:35 dist-test-slave-4d5t krb5kdc[3652](info): commencing operation
krb5kdc: starting...
W20260926 21:28:36.470593 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.007s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:36 dist-test-slave-4d5t krb5kdc[3652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:36.761327 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:37.115218 3665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:37.115610 3665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:37.115785 3665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:37.123538 3665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:37.123760 3665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:37.123876 3665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:37.123986 3665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:37.124076 3665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:37.134475 3665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:37.135937 3665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:37.137259 3665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:37.150211 3670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:37.150149 3671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:37.150174 3673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:37.150885 3665 server_base.cc:1034] running on GCE node
I20260926 21:28:37.151710 3665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:37.153360 3665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:37.154603 3665 hybrid_clock.cc:648] HybridClock initialized: now 1790458117154556 us; error 43 us; skew 500 ppm
I20260926 21:28:37.154973 3665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:37 dist-test-slave-4d5t krb5kdc[3652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:37.158324 3665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:37.159931 3665 webserver.cc:460] Webserver started at http://127.0.0.1:34859/ using document root <none> and password file <none>
I20260926 21:28:37.160681 3665 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:37.160823 3665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:37.161137 3665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:37.164346 3665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "a52eb00aba414d689d24eb21b5b6bf4f"
format_stamp: "Formatted at 2026-09-26 21:28:37 on dist-test-slave-4d5t"
server_key: "672630f58d40e4298e6b6b7f4babd90c"
server_key_iv: "03a1cecf79d5ae3662fb098cd70c29f8"
server_key_version: "encryptionkey@0"
I20260926 21:28:37.165278 3665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "a52eb00aba414d689d24eb21b5b6bf4f"
format_stamp: "Formatted at 2026-09-26 21:28:37 on dist-test-slave-4d5t"
server_key: "672630f58d40e4298e6b6b7f4babd90c"
server_key_iv: "03a1cecf79d5ae3662fb098cd70c29f8"
server_key_version: "encryptionkey@0"
I20260926 21:28:37.171118 3665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:28:37.175139 3680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:37.176659 3665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 21:28:37.176862 3665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "a52eb00aba414d689d24eb21b5b6bf4f"
format_stamp: "Formatted at 2026-09-26 21:28:37 on dist-test-slave-4d5t"
server_key: "672630f58d40e4298e6b6b7f4babd90c"
server_key_iv: "03a1cecf79d5ae3662fb098cd70c29f8"
server_key_version: "encryptionkey@0"
I20260926 21:28:37.177085 3665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:37.199172 3665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:37.204212 3665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:37.204666 3665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:37.226594 3665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33017
I20260926 21:28:37.226701 3732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33017 every 8 connection(s)
I20260926 21:28:37.228037 3665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:37.230528 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3665
I20260926 21:28:37.230732 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:37.231273 14904 external_mini_cluster.cc:1428] Setting key 4d0c1adfa76ace03a44141556181f326
I20260926 21:28:37.232623 3733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:37.246737 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:37.233217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:36801 (local address 127.0.0.1:33017)
0926 21:28:37.234042 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:37.234055 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:37.234100 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:28:37.236148 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:37.236171 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:37.236685 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:37.237085 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:37.237970 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:37.239356 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:37.241595 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:37.242388 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:37.242577 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:37.244109 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:37.244117 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:37.244133 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:37.244842 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:37.244936 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:37.244970 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:37.245171 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:37.245473 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 21:28:37.245669 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":376,"threads_started":1}
I20260926 21:28:37.248282 3733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f: Bootstrap starting.
I20260926 21:28:37.251243 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:37.251832 3733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:37.252951 3733 log.cc:824] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:37.257001 3733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f: No bootstrap required, opened a new log
I20260926 21:28:37.265892 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:37.256828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:53667 (local address 127.0.0.1:33017)
0926 21:28:37.257066 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:37.257074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:37.257368 (+ 294us) server_negotiation.cc:407] Connection header received
0926 21:28:37.257519 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:37.257528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:37.257637 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:37.257790 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:37.258778 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:37.260155 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:37.261902 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:37.262716 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:37.262881 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:37.264429 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:37.264438 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:37.264445 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:37.264479 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:37.264703 (+ 224us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:37.264748 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:37.264935 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:37.265250 (+ 315us) server_negotiation.cc:299] Negotiation successful
0926 21:28:37.265331 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 21:28:37.268998 3733 raft_consensus.cc:348] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33017 } }
I20260926 21:28:37.269451 3733 raft_consensus.cc:374] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:37.269599 3733 raft_consensus.cc:729] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a52eb00aba414d689d24eb21b5b6bf4f, State: Initialized, Role: FOLLOWER
I20260926 21:28:37.270306 3733 consensus_queue.cc:260] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33017 } }
I20260926 21:28:37.270548 3733 raft_consensus.cc:388] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:37.270660 3733 raft_consensus.cc:482] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:37.270797 3733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:37.273284 3733 raft_consensus.cc:504] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33017 } }
I20260926 21:28:37.273840 3733 leader_election.cc:302] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a52eb00aba414d689d24eb21b5b6bf4f; no voters:
I20260926 21:28:37.274412 3733 leader_election.cc:288] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:37.274571 3743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:37.275319 3743 raft_consensus.cc:686] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [term 1 LEADER]: Becoming Leader. State: Replica: a52eb00aba414d689d24eb21b5b6bf4f, State: Running, Role: LEADER
I20260926 21:28:37.276003 3743 consensus_queue.cc:237] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33017 } }
I20260926 21:28:37.276813 3733 sys_catalog.cc:564] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:37.279294 3745 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a52eb00aba414d689d24eb21b5b6bf4f. Latest consensus state: current_term: 1 leader_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33017 } } }
I20260926 21:28:37.279848 3745 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:37.279278 3744 sys_catalog.cc:455] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a52eb00aba414d689d24eb21b5b6bf4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33017 } } }
I20260926 21:28:37.280098 3744 sys_catalog.cc:458] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:37.282064 3749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:37.287453 3749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:37.297063 3749 catalog_manager.cc:1347] Generated new cluster ID: a47cea0d2a304b45b5778355651c8270
I20260926 21:28:37.297256 3749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:37.308681 3749 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:37.310489 3749 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:37.317256 3749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a52eb00aba414d689d24eb21b5b6bf4f: Generated new TSK 0
I20260926 21:28:37.318166 3749 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:37.328410 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3665
W20260926 21:28:37.330672 3728 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:36801
W20260926 21:28:37.331471 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:37.331617 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3665
W20260926 21:28:37.349779 3650 connection.cc:535] client connection to 127.0.0.1:33017 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33017): Connection reset by peer (error 104)
2026-09-26T21:28:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2042 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:37 dist-test-slave-4d5t krb5kdc[3764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:37 dist-test-slave-4d5t krb5kdc[3764](info): set up 2 sockets
Sep 26 21:28:37 dist-test-slave-4d5t krb5kdc[3764](info): commencing operation
krb5kdc: starting...
W20260926 21:28:38.493525 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:38 dist-test-slave-4d5t krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:38.792205 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:39.145915 3777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:39.146225 3777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:39.146384 3777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:39.153784 3777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:39.153910 3777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:39.154032 3777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:39.154114 3777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:39.154194 3777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:39.164640 3777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:39.166095 3777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:39.167452 3777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:39.179725 3783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:39.179729 3785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:39.179774 3782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:39.180799 3777 server_base.cc:1034] running on GCE node
I20260926 21:28:39.181696 3777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:39.183192 3777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:39.184429 3777 hybrid_clock.cc:648] HybridClock initialized: now 1790458119184374 us; error 42 us; skew 500 ppm
I20260926 21:28:39.184834 3777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:39 dist-test-slave-4d5t krb5kdc[3764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458119, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:39.192045 3777 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:39.193758 3777 webserver.cc:460] Webserver started at http://127.0.0.1:39073/ using document root <none> and password file <none>
I20260926 21:28:39.194512 3777 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:39.194631 3777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:39.194916 3777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:39.198365 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "b37e29329a774aa3a6eabdcbc8a17aa2"
format_stamp: "Formatted at 2026-09-26 21:28:39 on dist-test-slave-4d5t"
server_key: "332cff26a9464a85d9a6fc9404600936"
server_key_iv: "5bcdb5b198dc4231d921841e6ae5507e"
server_key_version: "encryptionkey@0"
I20260926 21:28:39.199188 3777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "b37e29329a774aa3a6eabdcbc8a17aa2"
format_stamp: "Formatted at 2026-09-26 21:28:39 on dist-test-slave-4d5t"
server_key: "332cff26a9464a85d9a6fc9404600936"
server_key_iv: "5bcdb5b198dc4231d921841e6ae5507e"
server_key_version: "encryptionkey@0"
I20260926 21:28:39.205093 3777 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 21:28:39.208768 3792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:39.210187 3777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:28:39.210350 3777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "b37e29329a774aa3a6eabdcbc8a17aa2"
format_stamp: "Formatted at 2026-09-26 21:28:39 on dist-test-slave-4d5t"
server_key: "332cff26a9464a85d9a6fc9404600936"
server_key_iv: "5bcdb5b198dc4231d921841e6ae5507e"
server_key_version: "encryptionkey@0"
I20260926 21:28:39.210559 3777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:39.217319 3777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:39.222043 3777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:39.222311 3777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:39.242528 3777 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43761
I20260926 21:28:39.242530 3843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43761 every 8 connection(s)
I20260926 21:28:39.244436 3777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:39.248317 3844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:39.251969 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3777
I20260926 21:28:39.252233 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:39.252909 14904 external_mini_cluster.cc:1428] Setting key 1906d50c836c60aff38cd6be2e4a231c
I20260926 21:28:39.261754 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:39.255116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:53849 (local address 127.0.0.1:43761)
0926 21:28:39.255927 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:39.255939 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:39.255982 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:28:39.257764 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:39.257790 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:39.258201 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:39.258586 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:39.259943 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:39.259951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:39.259966 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:39.259989 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:39.260067 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:39.260094 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:39.260329 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:39.260452 (+ 123us) server_negotiation.cc:299] Negotiation successful
0926 21:28:39.260717 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":383,"threads_started":1}
I20260926 21:28:39.262550 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2: Bootstrap starting.
I20260926 21:28:39.265807 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:39.266314 3844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:39.267412 3844 log.cc:824] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:39.270758 3844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2: No bootstrap required, opened a new log
I20260926 21:28:39.273893 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:39.270863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:51121 (local address 127.0.0.1:43761)
0926 21:28:39.271094 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:39.271101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:39.271509 (+ 408us) server_negotiation.cc:407] Connection header received
0926 21:28:39.271741 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:39.271745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:39.271798 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:39.271895 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:39.273107 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:39.273112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:39.273116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:39.273135 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:39.273182 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:39.273202 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:39.273333 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:39.273459 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 21:28:39.273516 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 21:28:39.280232 3844 raft_consensus.cc:348] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260926 21:28:39.280721 3844 raft_consensus.cc:374] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:39.280851 3844 raft_consensus.cc:729] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37e29329a774aa3a6eabdcbc8a17aa2, State: Initialized, Role: FOLLOWER
I20260926 21:28:39.281591 3844 consensus_queue.cc:260] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260926 21:28:39.281955 3844 raft_consensus.cc:388] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:39.282135 3844 raft_consensus.cc:482] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:39.282327 3844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:39.284933 3844 raft_consensus.cc:504] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260926 21:28:39.285480 3844 leader_election.cc:302] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b37e29329a774aa3a6eabdcbc8a17aa2; no voters:
I20260926 21:28:39.286104 3844 leader_election.cc:288] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:39.286295 3854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:39.287006 3854 raft_consensus.cc:686] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [term 1 LEADER]: Becoming Leader. State: Replica: b37e29329a774aa3a6eabdcbc8a17aa2, State: Running, Role: LEADER
I20260926 21:28:39.287598 3854 consensus_queue.cc:237] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } }
I20260926 21:28:39.288407 3844 sys_catalog.cc:564] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:39.290850 3855 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } } }
I20260926 21:28:39.290684 3856 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b37e29329a774aa3a6eabdcbc8a17aa2. Latest consensus state: current_term: 1 leader_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37e29329a774aa3a6eabdcbc8a17aa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43761 } } }
I20260926 21:28:39.291337 3855 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:39.291349 3856 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:39.293776 3861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:39.299196 3861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:39.309198 3861 catalog_manager.cc:1347] Generated new cluster ID: 10045126ea0242cf8f451320986112fc
I20260926 21:28:39.309360 3861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:39.328361 3861 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:39.328831 3861 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:39.339289 3861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b37e29329a774aa3a6eabdcbc8a17aa2: Generated new TSK 0
I20260926 21:28:39.340160 3861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:39.347919 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3777
W20260926 21:28:39.349038 3839 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:53849
W20260926 21:28:39.349752 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3777: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:39.349910 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3777
2026-09-26T21:28:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2014 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:39 dist-test-slave-4d5t krb5kdc[3875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:39 dist-test-slave-4d5t krb5kdc[3875](info): set up 2 sockets
Sep 26 21:28:39 dist-test-slave-4d5t krb5kdc[3875](info): commencing operation
krb5kdc: starting...
W20260926 21:28:40.497778 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:40 dist-test-slave-4d5t krb5kdc[3875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:40.800247 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:41.150034 3888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:41.150348 3888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:41.150516 3888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:41.157964 3888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:41.158100 3888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:41.158217 3888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:41.158300 3888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:41.158385 3888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:41.168568 3888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:41.169952 3888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:41.171257 3888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:41.183249 3893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:41.183230 3894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:41.183296 3896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:41.184618 3888 server_base.cc:1034] running on GCE node
I20260926 21:28:41.185457 3888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:41.186939 3888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:41.188200 3888 hybrid_clock.cc:648] HybridClock initialized: now 1790458121188108 us; error 47 us; skew 500 ppm
I20260926 21:28:41.188670 3888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:41 dist-test-slave-4d5t krb5kdc[3875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458121, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:41.191881 3888 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:41.193442 3888 webserver.cc:460] Webserver started at http://127.0.0.1:44167/ using document root <none> and password file <none>
I20260926 21:28:41.194108 3888 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:41.194212 3888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:41.194453 3888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:41.197484 3888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "1289604222f442fd8e9852a52e2f4e9d"
format_stamp: "Formatted at 2026-09-26 21:28:41 on dist-test-slave-4d5t"
server_key: "decaca0330c09a98b2124e4cfb696fbc"
server_key_iv: "d9df4b35de7ad3126bdcef78caa80055"
server_key_version: "encryptionkey@0"
I20260926 21:28:41.198199 3888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "1289604222f442fd8e9852a52e2f4e9d"
format_stamp: "Formatted at 2026-09-26 21:28:41 on dist-test-slave-4d5t"
server_key: "decaca0330c09a98b2124e4cfb696fbc"
server_key_iv: "d9df4b35de7ad3126bdcef78caa80055"
server_key_version: "encryptionkey@0"
I20260926 21:28:41.203565 3888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 21:28:41.207453 3903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:41.209000 3888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 21:28:41.209203 3888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "1289604222f442fd8e9852a52e2f4e9d"
format_stamp: "Formatted at 2026-09-26 21:28:41 on dist-test-slave-4d5t"
server_key: "decaca0330c09a98b2124e4cfb696fbc"
server_key_iv: "d9df4b35de7ad3126bdcef78caa80055"
server_key_version: "encryptionkey@0"
I20260926 21:28:41.209405 3888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:41.223274 3888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:41.228197 3888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:41.228667 3888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:41.249848 3888 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44165
I20260926 21:28:41.249851 3954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44165 every 8 connection(s)
I20260926 21:28:41.251366 3888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:41.255175 3955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:41.260349 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 3888
I20260926 21:28:41.260605 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:41.261145 14904 external_mini_cluster.cc:1428] Setting key f4e0e0291aeab0b298386466d1434596
I20260926 21:28:41.267299 3955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d: Bootstrap starting.
I20260926 21:28:41.270711 3955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:41.271823 3955 log.cc:824] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:41.274998 3955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d: No bootstrap required, opened a new log
Sep 26 21:28:41 dist-test-slave-4d5t krb5kdc[3875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 21:28:41.283561 3955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1289604222f442fd8e9852a52e2f4e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260926 21:28:41.284049 3955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:41.284164 3955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1289604222f442fd8e9852a52e2f4e9d, State: Initialized, Role: FOLLOWER
I20260926 21:28:41.284857 3955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1289604222f442fd8e9852a52e2f4e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260926 21:28:41.285140 3955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:41.285290 3955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:41.285462 3955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:41.288091 3955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1289604222f442fd8e9852a52e2f4e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260926 21:28:41.288640 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:41.263214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:44899 (local address 127.0.0.1:44165)
0926 21:28:41.264430 (+ 1216us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:41.264450 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:41.264521 (+ 71us) server_negotiation.cc:407] Connection header received
0926 21:28:41.267144 (+ 2623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:41.267182 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:41.267885 (+ 703us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:41.268359 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:41.270109 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:41.271679 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:41.273587 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:41.274336 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:41.274553 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:41.279263 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:41.279273 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:41.279296 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:41.279352 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:41.283668 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:41.284923 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:41.284931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:41.284938 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:41.285185 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:41.285702 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:41.285706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:41.285708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:41.286290 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:41.286666 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:41.287168 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 21:28:41.287502 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":494,"threads_started":1}
I20260926 21:28:41.288672 3955 leader_election.cc:302] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1289604222f442fd8e9852a52e2f4e9d; no voters:
I20260926 21:28:41.289451 3955 leader_election.cc:288] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:41.289595 3960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:41.290648 3960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [term 1 LEADER]: Becoming Leader. State: Replica: 1289604222f442fd8e9852a52e2f4e9d, State: Running, Role: LEADER
I20260926 21:28:41.291254 3960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1289604222f442fd8e9852a52e2f4e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260926 21:28:41.291683 3955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:41.294242 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:41.300812 3961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1289604222f442fd8e9852a52e2f4e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1289604222f442fd8e9852a52e2f4e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } } }
I20260926 21:28:41.301407 3961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:41.304575 3962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1289604222f442fd8e9852a52e2f4e9d. Latest consensus state: current_term: 1 leader_uuid: "1289604222f442fd8e9852a52e2f4e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1289604222f442fd8e9852a52e2f4e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } } }
I20260926 21:28:41.305203 3962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:41.309211 3969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:41.316241 3969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:41.326552 3969 catalog_manager.cc:1347] Generated new cluster ID: 100e5e44225c4c7e9d78d4530c0f95cb
I20260926 21:28:41.326675 3969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:41.331471 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:41.312204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:42105 (local address 127.0.0.1:44165)
0926 21:28:41.312388 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:41.312396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:41.312424 (+ 28us) server_negotiation.cc:407] Connection header received
0926 21:28:41.312689 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:41.312695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:41.312783 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:41.312923 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:41.314754 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:41.315604 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:41.317748 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:41.318756 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:41.318905 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:41.323575 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:41.323584 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:41.323589 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:41.323897 (+ 308us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:41.327754 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:41.329180 (+ 1426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:41.329186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:41.329194 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:41.329348 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:41.329840 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:41.329845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:41.329848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:41.330231 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:41.330476 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:41.330891 (+ 415us) server_negotiation.cc:299] Negotiation successful
0926 21:28:41.331070 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":42}
I20260926 21:28:41.336676 3969 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:41.337905 3969 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:41.345819 3969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1289604222f442fd8e9852a52e2f4e9d: Generated new TSK 0
I20260926 21:28:41.346863 3969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:41.360339 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 3888
I20260926 21:28:41.361620 3950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:41.455621 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 3888
W20260926 21:28:41.474298 3873 connection.cc:535] client connection to 127.0.0.1:44165 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44165): Connection reset by peer (error 104)
2026-09-26T21:28:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2114 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:41 dist-test-slave-4d5t krb5kdc[3988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:41 dist-test-slave-4d5t krb5kdc[3988](info): set up 2 sockets
Sep 26 21:28:41 dist-test-slave-4d5t krb5kdc[3988](info): commencing operation
krb5kdc: starting...
W20260926 21:28:42.642513 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.005s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:42 dist-test-slave-4d5t krb5kdc[3988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:42.944860 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:43.297801 4001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:43.298123 4001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:43.298218 4001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:43.305863 4001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:43.305997 4001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:43.306043 4001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:43.306098 4001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:43.306144 4001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:43.316248 4001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44787
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:43.317685 4001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:43.318975 4001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:43.331645 4006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:43.331741 4007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:43.331718 4009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:43.333024 4001 server_base.cc:1034] running on GCE node
I20260926 21:28:43.333971 4001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:43.335587 4001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:43.336846 4001 hybrid_clock.cc:648] HybridClock initialized: now 1790458123336802 us; error 36 us; skew 500 ppm
I20260926 21:28:43.337251 4001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:43 dist-test-slave-4d5t krb5kdc[3988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458123, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:43.341603 4001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:43.343348 4001 webserver.cc:460] Webserver started at http://127.0.0.1:34397/ using document root <none> and password file <none>
I20260926 21:28:43.344154 4001 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:43.344316 4001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:43.344730 4001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:43.348021 4001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "a93b9725ffd64997bd2b9c2debe43fec"
format_stamp: "Formatted at 2026-09-26 21:28:43 on dist-test-slave-4d5t"
server_key: "68af3cd4fca0e51b07a6bd57f699ee58"
server_key_iv: "685272e7ac7b97565b07bd6d621cdab3"
server_key_version: "encryptionkey@0"
I20260926 21:28:43.348968 4001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "a93b9725ffd64997bd2b9c2debe43fec"
format_stamp: "Formatted at 2026-09-26 21:28:43 on dist-test-slave-4d5t"
server_key: "68af3cd4fca0e51b07a6bd57f699ee58"
server_key_iv: "685272e7ac7b97565b07bd6d621cdab3"
server_key_version: "encryptionkey@0"
I20260926 21:28:43.354852 4001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 21:28:43.358976 4016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:43.360795 4001 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:28:43.361006 4001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "a93b9725ffd64997bd2b9c2debe43fec"
format_stamp: "Formatted at 2026-09-26 21:28:43 on dist-test-slave-4d5t"
server_key: "68af3cd4fca0e51b07a6bd57f699ee58"
server_key_iv: "685272e7ac7b97565b07bd6d621cdab3"
server_key_version: "encryptionkey@0"
I20260926 21:28:43.361243 4001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:43.377750 4001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:43.382779 4001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:43.383131 4001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:43.406430 4001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44787
I20260926 21:28:43.406502 4068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44787 every 8 connection(s)
I20260926 21:28:43.407995 4001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:43.412053 4069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:43.414191 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4001
I20260926 21:28:43.414389 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:43.414956 14904 external_mini_cluster.cc:1428] Setting key 428516fed68acf312d8c977ddcb3c472
I20260926 21:28:43.426765 4069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec: Bootstrap starting.
I20260926 21:28:43.428470 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:43.416932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:49343 (local address 127.0.0.1:44787)
0926 21:28:43.417733 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:43.417745 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:43.417797 (+ 52us) server_negotiation.cc:407] Connection header received
0926 21:28:43.419391 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:43.419416 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:43.419827 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:43.420227 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:43.420775 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:43.421989 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:43.423309 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:43.424153 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:43.424333 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:43.425759 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:43.425766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:43.425780 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:43.426313 (+ 533us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:43.426401 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:43.426430 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:43.426562 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:43.427286 (+ 724us) server_negotiation.cc:299] Negotiation successful
0926 21:28:43.427458 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":360,"threads_started":1}
I20260926 21:28:43.430437 4069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:43.432050 4069 log.cc:824] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:43.432869 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:43.435474 4069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec: No bootstrap required, opened a new log
I20260926 21:28:43.444850 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:43.437592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:57607 (local address 127.0.0.1:44787)
0926 21:28:43.437791 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:43.437805 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:43.438321 (+ 516us) server_negotiation.cc:407] Connection header received
0926 21:28:43.438592 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:43.438596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:43.438684 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:43.438807 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:43.439407 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:43.440764 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:43.442122 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:43.442619 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:43.442767 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:43.444016 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:43.444037 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:43.444040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:43.444060 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:43.444117 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:43.444133 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:43.444268 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:43.444461 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 21:28:43.444520 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 21:28:43.445436 4069 raft_consensus.cc:348] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93b9725ffd64997bd2b9c2debe43fec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260926 21:28:43.445874 4069 raft_consensus.cc:374] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:43.445999 4069 raft_consensus.cc:729] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93b9725ffd64997bd2b9c2debe43fec, State: Initialized, Role: FOLLOWER
I20260926 21:28:43.446650 4069 consensus_queue.cc:260] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93b9725ffd64997bd2b9c2debe43fec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260926 21:28:43.446926 4069 raft_consensus.cc:388] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:43.447077 4069 raft_consensus.cc:482] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:43.447261 4069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:43.449635 4069 raft_consensus.cc:504] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93b9725ffd64997bd2b9c2debe43fec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260926 21:28:43.450169 4069 leader_election.cc:302] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a93b9725ffd64997bd2b9c2debe43fec; no voters:
I20260926 21:28:43.450829 4069 leader_election.cc:288] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:43.450985 4079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:43.451743 4079 raft_consensus.cc:686] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [term 1 LEADER]: Becoming Leader. State: Replica: a93b9725ffd64997bd2b9c2debe43fec, State: Running, Role: LEADER
I20260926 21:28:43.452363 4079 consensus_queue.cc:237] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93b9725ffd64997bd2b9c2debe43fec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } }
I20260926 21:28:43.453379 4069 sys_catalog.cc:564] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:43.455479 4080 sys_catalog.cc:455] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a93b9725ffd64997bd2b9c2debe43fec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93b9725ffd64997bd2b9c2debe43fec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } } }
I20260926 21:28:43.455964 4080 sys_catalog.cc:458] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:43.456877 4081 sys_catalog.cc:455] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [sys.catalog]: SysCatalogTable state changed. Reason: New leader a93b9725ffd64997bd2b9c2debe43fec. Latest consensus state: current_term: 1 leader_uuid: "a93b9725ffd64997bd2b9c2debe43fec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a93b9725ffd64997bd2b9c2debe43fec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44787 } } }
I20260926 21:28:43.457312 4081 sys_catalog.cc:458] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:43.459842 4086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:43.465723 4086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:43.476507 4086 catalog_manager.cc:1347] Generated new cluster ID: ca05accee5404cc98a2307b60758174d
I20260926 21:28:43.476930 4086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:43.501327 4086 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:43.502803 4086 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:43.515484 4086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a93b9725ffd64997bd2b9c2debe43fec: Generated new TSK 0
I20260926 21:28:43.516451 4086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:43.523948 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4001
W20260926 21:28:43.525494 4064 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:49343
W20260926 21:28:43.526363 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:43.526515 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4001
W20260926 21:28:43.544736 3986 connection.cc:535] client connection to 127.0.0.1:44787 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44787): Connection reset by peer (error 104)
2026-09-26T21:28:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:43 dist-test-slave-4d5t krb5kdc[4100](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:43 dist-test-slave-4d5t krb5kdc[4100](info): set up 2 sockets
Sep 26 21:28:43 dist-test-slave-4d5t krb5kdc[4100](info): commencing operation
krb5kdc: starting...
W20260926 21:28:44.661532 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:44 dist-test-slave-4d5t krb5kdc[4100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 21:28:44.959661 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:45.320747 4113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:45.321069 4113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:45.321165 4113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:45.328639 4113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:45.328766 4113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:45.328814 4113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:45.328869 4113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:45.328914 4113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:45.339180 4113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:45.340534 4113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:45.341924 4113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:45.354254 4121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:45.354310 4118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:45.354337 4119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:45.355738 4113 server_base.cc:1034] running on GCE node
I20260926 21:28:45.356705 4113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:45.358484 4113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:45.359789 4113 hybrid_clock.cc:648] HybridClock initialized: now 1790458125359729 us; error 47 us; skew 500 ppm
I20260926 21:28:45.360215 4113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:45 dist-test-slave-4d5t krb5kdc[4100](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458125, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:45.371682 4113 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 21:28:45.373471 4113 webserver.cc:460] Webserver started at http://127.0.0.1:34153/ using document root <none> and password file <none>
I20260926 21:28:45.374167 4113 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:45.374277 4113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:45.374542 4113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:45.377720 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "8d7a680267af46e9a1fe929e3292d104"
format_stamp: "Formatted at 2026-09-26 21:28:45 on dist-test-slave-4d5t"
server_key: "493a4241758a6a9e2a770a95a118eee4"
server_key_iv: "14b5756c65ed012488389d43188b07f2"
server_key_version: "encryptionkey@0"
I20260926 21:28:45.378459 4113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "8d7a680267af46e9a1fe929e3292d104"
format_stamp: "Formatted at 2026-09-26 21:28:45 on dist-test-slave-4d5t"
server_key: "493a4241758a6a9e2a770a95a118eee4"
server_key_iv: "14b5756c65ed012488389d43188b07f2"
server_key_version: "encryptionkey@0"
I20260926 21:28:45.383862 4113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 21:28:45.388084 4128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:45.389833 4113 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 21:28:45.390008 4113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "8d7a680267af46e9a1fe929e3292d104"
format_stamp: "Formatted at 2026-09-26 21:28:45 on dist-test-slave-4d5t"
server_key: "493a4241758a6a9e2a770a95a118eee4"
server_key_iv: "14b5756c65ed012488389d43188b07f2"
server_key_version: "encryptionkey@0"
I20260926 21:28:45.390208 4113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:45.396870 4113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:45.401434 4113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:45.401710 4113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:45.422515 4113 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45259
I20260926 21:28:45.422575 4179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45259 every 8 connection(s)
I20260926 21:28:45.424002 4113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:45.427706 4180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:45.430300 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4113
I20260926 21:28:45.430492 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:45.431025 14904 external_mini_cluster.cc:1428] Setting key 6310686b5fa040b4005d20bf8b32c4ce
I20260926 21:28:45.439898 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:45.433423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:45481 (local address 127.0.0.1:45259)
0926 21:28:45.434338 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:45.434350 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:45.434394 (+ 44us) server_negotiation.cc:407] Connection header received
0926 21:28:45.436234 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:45.436258 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:45.436729 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:45.437105 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:45.438206 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:45.438213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:45.438226 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:45.438245 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:45.438302 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:45.438322 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:45.438522 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:45.438634 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 21:28:45.438853 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":452,"threads_started":1}
I20260926 21:28:45.440724 4180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104: Bootstrap starting.
I20260926 21:28:45.443871 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:45.443984 4180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:45.445268 4180 log.cc:824] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:45.449002 4180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104: No bootstrap required, opened a new log
I20260926 21:28:45.453459 4183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:45.450669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.115:37627 (local address 127.0.0.1:45259)
0926 21:28:45.450843 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:45.450876 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:45.451384 (+ 508us) server_negotiation.cc:407] Connection header received
0926 21:28:45.451492 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:45.451497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:45.451565 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:45.451687 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:45.452773 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:45.452779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:45.452784 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:45.452816 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:45.452869 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:45.452888 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:45.453022 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:45.453101 (+ 79us) server_negotiation.cc:299] Negotiation successful
0926 21:28:45.453165 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 21:28:45.458588 4180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7a680267af46e9a1fe929e3292d104" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45259 } }
I20260926 21:28:45.458979 4180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:45.459054 4180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7a680267af46e9a1fe929e3292d104, State: Initialized, Role: FOLLOWER
I20260926 21:28:45.459631 4180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7a680267af46e9a1fe929e3292d104" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45259 } }
I20260926 21:28:45.459842 4180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:45.459949 4180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:45.460068 4180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:45.462394 4180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7a680267af46e9a1fe929e3292d104" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45259 } }
I20260926 21:28:45.463047 4180 leader_election.cc:302] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d7a680267af46e9a1fe929e3292d104; no voters:
I20260926 21:28:45.463726 4180 leader_election.cc:288] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:45.463963 4190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:45.464767 4190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [term 1 LEADER]: Becoming Leader. State: Replica: 8d7a680267af46e9a1fe929e3292d104, State: Running, Role: LEADER
I20260926 21:28:45.465488 4190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7a680267af46e9a1fe929e3292d104" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45259 } }
I20260926 21:28:45.465919 4180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:45.468536 4191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d7a680267af46e9a1fe929e3292d104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7a680267af46e9a1fe929e3292d104" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45259 } } }
I20260926 21:28:45.469044 4191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:45.469802 4192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d7a680267af46e9a1fe929e3292d104. Latest consensus state: current_term: 1 leader_uuid: "8d7a680267af46e9a1fe929e3292d104" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7a680267af46e9a1fe929e3292d104" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45259 } } }
I20260926 21:28:45.470515 4192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:45.471786 4197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:45.477735 4197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:45.487797 4197 catalog_manager.cc:1347] Generated new cluster ID: 2d89192055ee4bfe9e0355c997e5575f
I20260926 21:28:45.487967 4197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:45.496155 4197 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:45.496639 4197 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:45.504194 4197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d7a680267af46e9a1fe929e3292d104: Generated new TSK 0
I20260926 21:28:45.505175 4197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:45.517681 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4113
W20260926 21:28:45.518877 4175 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.115:45481
W20260926 21:28:45.519626 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4113: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:45.519776 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4113
2026-09-26T21:28:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1991 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:45 dist-test-slave-4d5t krb5kdc[4211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:45 dist-test-slave-4d5t krb5kdc[4211](info): set up 2 sockets
Sep 26 21:28:45 dist-test-slave-4d5t krb5kdc[4211](info): commencing operation
krb5kdc: starting...
W20260926 21:28:46.662088 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:46 dist-test-slave-4d5t krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:46.956035 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41917
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:41917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:47.317124 4224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:47.317445 4224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:47.317616 4224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:47.325402 4224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:47.325531 4224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:47.325611 4224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:47.325731 4224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:47.325794 4224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:47.336113 4224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:41917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:41917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:47.337531 4224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:47.338851 4224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:47.351033 4229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:47.351052 4230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:47.351212 4232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:47.352329 4224 server_base.cc:1034] running on GCE node
I20260926 21:28:47.353286 4224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:47.355119 4224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:47.356405 4224 hybrid_clock.cc:648] HybridClock initialized: now 1790458127356318 us; error 45 us; skew 500 ppm
I20260926 21:28:47.356861 4224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:47 dist-test-slave-4d5t krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458127, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:47.360862 4224 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:47.362668 4224 webserver.cc:460] Webserver started at http://127.14.142.62:35287/ using document root <none> and password file <none>
I20260926 21:28:47.363453 4224 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:47.363631 4224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:47.363976 4224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:47.367328 4224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "e3640510c56c42d493c254a67bc56d02"
format_stamp: "Formatted at 2026-09-26 21:28:47 on dist-test-slave-4d5t"
server_key: "24284536554e8f24dcfa312a3934b813"
server_key_iv: "5d48fbb6d91de14118dfa3a647fb1fab"
server_key_version: "encryptionkey@0"
I20260926 21:28:47.368192 4224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "e3640510c56c42d493c254a67bc56d02"
format_stamp: "Formatted at 2026-09-26 21:28:47 on dist-test-slave-4d5t"
server_key: "24284536554e8f24dcfa312a3934b813"
server_key_iv: "5d48fbb6d91de14118dfa3a647fb1fab"
server_key_version: "encryptionkey@0"
I20260926 21:28:47.374325 4224 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260926 21:28:47.379281 4240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:47.381359 4224 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 21:28:47.381639 4224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "e3640510c56c42d493c254a67bc56d02"
format_stamp: "Formatted at 2026-09-26 21:28:47 on dist-test-slave-4d5t"
server_key: "24284536554e8f24dcfa312a3934b813"
server_key_iv: "5d48fbb6d91de14118dfa3a647fb1fab"
server_key_version: "encryptionkey@0"
I20260926 21:28:47.381904 4224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:47.410905 4224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:47.416141 4224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:47.416501 4224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:47.437897 4224 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:41917
I20260926 21:28:47.437923 4291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:41917 every 8 connection(s)
I20260926 21:28:47.439481 4224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:47.443850 4292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:47.446518 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4224
I20260926 21:28:47.446709 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:47.447186 14904 external_mini_cluster.cc:1428] Setting key 0e026f1c7f64a50ef6d01b00131e9239
I20260926 21:28:47.458607 4292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02: Bootstrap starting.
I20260926 21:28:47.463338 4292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02: Neither blocks nor log segments found. Creating new log.
Sep 26 21:28:47 dist-test-slave-4d5t krb5kdc[4211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:47.464809 4292 log.cc:824] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:47.468204 4292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02: No bootstrap required, opened a new log
I20260926 21:28:47.473783 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:47.449116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54893 (local address 127.14.142.62:41917)
0926 21:28:47.450004 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:47.450017 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:47.450058 (+ 41us) server_negotiation.cc:407] Connection header received
0926 21:28:47.451749 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:47.451773 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:47.452197 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:47.452543 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:47.454752 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:47.457325 (+ 2573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:47.459337 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:47.460059 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:47.460203 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:47.465836 (+ 5633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:47.465845 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:47.465865 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:47.465906 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:47.469282 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:47.470232 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:47.470239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:47.470245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:47.470408 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:47.470911 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:47.470919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:47.470922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:47.471468 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:47.471866 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:47.472284 (+ 418us) server_negotiation.cc:299] Negotiation successful
0926 21:28:47.472683 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":422,"threads_started":1}
I20260926 21:28:47.478161 4292 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3640510c56c42d493c254a67bc56d02" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41917 } }
I20260926 21:28:47.478734 4292 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:47.478780 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:47.478883 4292 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3640510c56c42d493c254a67bc56d02, State: Initialized, Role: FOLLOWER
I20260926 21:28:47.479777 4292 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3640510c56c42d493c254a67bc56d02" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41917 } }
I20260926 21:28:47.480068 4292 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:47.480237 4292 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:47.480419 4292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:47.482976 4292 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3640510c56c42d493c254a67bc56d02" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41917 } }
I20260926 21:28:47.483667 4292 leader_election.cc:302] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3640510c56c42d493c254a67bc56d02; no voters:
I20260926 21:28:47.484438 4292 leader_election.cc:288] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:47.484532 4302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:47.485239 4302 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [term 1 LEADER]: Becoming Leader. State: Replica: e3640510c56c42d493c254a67bc56d02, State: Running, Role: LEADER
I20260926 21:28:47.486267 4302 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3640510c56c42d493c254a67bc56d02" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41917 } }
I20260926 21:28:47.488793 4292 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:47.489437 4304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3640510c56c42d493c254a67bc56d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3640510c56c42d493c254a67bc56d02" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41917 } } }
I20260926 21:28:47.489869 4304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:47.507510 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:47.483510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38295 (local address 127.14.142.62:41917)
0926 21:28:47.483769 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:47.483778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:47.484338 (+ 560us) server_negotiation.cc:407] Connection header received
0926 21:28:47.484574 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:47.484580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:47.484663 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:47.484792 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:47.486564 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:47.487662 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:47.489456 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:47.490156 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:47.490286 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:47.495048 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:47.495059 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:47.495068 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:47.495134 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:47.500290 (+ 5156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:47.502590 (+ 2300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:47.502602 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:47.502609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:47.502930 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:47.503793 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:47.503801 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:47.503807 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:47.504407 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:47.504860 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:47.506558 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0926 21:28:47.506892 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 21:28:47.512199 4306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:47.512362 4303 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3640510c56c42d493c254a67bc56d02. Latest consensus state: current_term: 1 leader_uuid: "e3640510c56c42d493c254a67bc56d02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3640510c56c42d493c254a67bc56d02" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 41917 } } }
I20260926 21:28:47.512872 4303 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:47.519194 4306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:47.530534 4306 catalog_manager.cc:1347] Generated new cluster ID: d708d165a9ef426191e796bdc316b1f5
I20260926 21:28:47.530680 4306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:47.549636 4306 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:47.551756 4306 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:47.565152 4306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3640510c56c42d493c254a67bc56d02: Generated new TSK 0
I20260926 21:28:47.566108 4306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:47.579564 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4224
I20260926 21:28:47.580859 4287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:47.676278 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4224
W20260926 21:28:47.700739 4209 connection.cc:535] client connection to 127.14.142.62:41917 recv error: Network error: failed to read from TLS socket (remote: 127.14.142.62:41917): Connection reset by peer (error 104)
2026-09-26T21:28:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2163 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:47 dist-test-slave-4d5t krb5kdc[4325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:47 dist-test-slave-4d5t krb5kdc[4325](info): set up 2 sockets
Sep 26 21:28:47 dist-test-slave-4d5t krb5kdc[4325](info): commencing operation
krb5kdc: starting...
W20260926 21:28:48.825661 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:49 dist-test-slave-4d5t krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:49.125072 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:33433
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:33433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:49.495005 4338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:49.495313 4338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:49.495410 4338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:49.502868 4338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:49.502995 4338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:49.503043 4338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:49.503110 4338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:49.503156 4338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:49.513325 4338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:33433
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:33433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:49.514652 4338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:49.515991 4338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:49.528465 4343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:49.528494 4344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:49.528664 4346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:49.529958 4338 server_base.cc:1034] running on GCE node
I20260926 21:28:49.530854 4338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:49.532662 4338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:49.533958 4338 hybrid_clock.cc:648] HybridClock initialized: now 1790458129533879 us; error 53 us; skew 500 ppm
I20260926 21:28:49.534376 4338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:49 dist-test-slave-4d5t krb5kdc[4325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458129, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:49.537906 4338 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:49.539527 4338 webserver.cc:460] Webserver started at http://127.14.142.62:45619/ using document root <none> and password file <none>
I20260926 21:28:49.540215 4338 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:49.540320 4338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:49.540634 4338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:49.543633 4338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "1c2f778d1255474189a5a974acf9c70c"
format_stamp: "Formatted at 2026-09-26 21:28:49 on dist-test-slave-4d5t"
server_key: "39524e8ec82993e70b0788e87ad04a88"
server_key_iv: "742a1628702a76f0df3bd65cc3def5b7"
server_key_version: "encryptionkey@0"
I20260926 21:28:49.544428 4338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "1c2f778d1255474189a5a974acf9c70c"
format_stamp: "Formatted at 2026-09-26 21:28:49 on dist-test-slave-4d5t"
server_key: "39524e8ec82993e70b0788e87ad04a88"
server_key_iv: "742a1628702a76f0df3bd65cc3def5b7"
server_key_version: "encryptionkey@0"
I20260926 21:28:49.549738 4338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 21:28:49.553496 4353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:49.555078 4338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 21:28:49.555251 4338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "1c2f778d1255474189a5a974acf9c70c"
format_stamp: "Formatted at 2026-09-26 21:28:49 on dist-test-slave-4d5t"
server_key: "39524e8ec82993e70b0788e87ad04a88"
server_key_iv: "742a1628702a76f0df3bd65cc3def5b7"
server_key_version: "encryptionkey@0"
I20260926 21:28:49.555447 4338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:49.581032 4338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:49.585814 4338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:49.586097 4338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:49.608021 4338 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:33433
I20260926 21:28:49.608052 4404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:33433 every 8 connection(s)
I20260926 21:28:49.609637 4338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:49.613528 4405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:49.615835 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4338
I20260926 21:28:49.616045 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:49.616519 14904 external_mini_cluster.cc:1428] Setting key 137864a4e203b9cd212da2c250fa60a2
I20260926 21:28:49.628788 4405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c: Bootstrap starting.
I20260926 21:28:49.630398 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:49.618477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42449 (local address 127.14.142.62:33433)
0926 21:28:49.619354 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:49.619369 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:49.619415 (+ 46us) server_negotiation.cc:407] Connection header received
0926 21:28:49.621139 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:49.621163 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:49.621594 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:49.621927 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:49.622531 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:49.623776 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:49.625732 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:49.626443 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:49.626580 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:49.627981 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:49.627988 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:49.628002 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:49.628638 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:49.628719 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:49.628744 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:49.628873 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:49.629199 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 21:28:49.629403 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":420,"threads_started":1}
I20260926 21:28:49.632447 4405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:49.633798 4405 log.cc:824] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:49.634622 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:49.636963 4405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c: No bootstrap required, opened a new log
I20260926 21:28:49.647635 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:49.639585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47485 (local address 127.14.142.62:33433)
0926 21:28:49.639932 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:49.639942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:49.640463 (+ 521us) server_negotiation.cc:407] Connection header received
0926 21:28:49.640709 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:49.640714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:49.640776 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:49.640855 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:49.641561 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:49.642463 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:49.644800 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:49.645280 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:49.645395 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:49.646788 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:49.646793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:49.646796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:49.646818 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:49.646876 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:49.646892 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:49.646989 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:49.647209 (+ 220us) server_negotiation.cc:299] Negotiation successful
0926 21:28:49.647253 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260926 21:28:49.648079 4405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2f778d1255474189a5a974acf9c70c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 33433 } }
I20260926 21:28:49.648663 4405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:49.648787 4405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2f778d1255474189a5a974acf9c70c, State: Initialized, Role: FOLLOWER
I20260926 21:28:49.649497 4405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2f778d1255474189a5a974acf9c70c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 33433 } }
I20260926 21:28:49.649745 4405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:49.649856 4405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:49.649998 4405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:49.653010 4405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2f778d1255474189a5a974acf9c70c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 33433 } }
I20260926 21:28:49.653540 4405 leader_election.cc:302] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c2f778d1255474189a5a974acf9c70c; no voters:
I20260926 21:28:49.654155 4405 leader_election.cc:288] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:49.654320 4415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:49.655812 4415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [term 1 LEADER]: Becoming Leader. State: Replica: 1c2f778d1255474189a5a974acf9c70c, State: Running, Role: LEADER
I20260926 21:28:49.656424 4415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2f778d1255474189a5a974acf9c70c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 33433 } }
I20260926 21:28:49.656741 4405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:49.659554 4417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c2f778d1255474189a5a974acf9c70c. Latest consensus state: current_term: 1 leader_uuid: "1c2f778d1255474189a5a974acf9c70c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2f778d1255474189a5a974acf9c70c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 33433 } } }
I20260926 21:28:49.659556 4416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c2f778d1255474189a5a974acf9c70c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2f778d1255474189a5a974acf9c70c" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 33433 } } }
I20260926 21:28:49.660081 4417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:49.660148 4416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:49.663182 4423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:49.669632 4423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:49.680917 4423 catalog_manager.cc:1347] Generated new cluster ID: 582f8420fd1544e2a64ed8d9fbdf1666
I20260926 21:28:49.681047 4423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:49.702901 4423 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:49.704892 4423 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:49.712049 4423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c2f778d1255474189a5a974acf9c70c: Generated new TSK 0
I20260926 21:28:49.713608 4423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:49.730531 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4338
W20260926 21:28:49.732147 4400 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42449
W20260926 21:28:49.733000 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4338: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:49.733187 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4338
W20260926 21:28:49.751247 4323 connection.cc:535] client connection to 127.14.142.62:33433 recv error: Network error: failed to read from TLS socket (remote: 127.14.142.62:33433): Connection reset by peer (error 104)
2026-09-26T21:28:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2049 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:49 dist-test-slave-4d5t krb5kdc[4436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:49 dist-test-slave-4d5t krb5kdc[4436](info): set up 2 sockets
Sep 26 21:28:49 dist-test-slave-4d5t krb5kdc[4436](info): commencing operation
krb5kdc: starting...
W20260926 21:28:50.886031 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:51 dist-test-slave-4d5t krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:51.189142 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44863
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:51.540591 4449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:51.540890 4449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:51.541057 4449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:51.548591 4449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:51.548707 4449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:51.548852 4449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:51.548945 4449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:51.549031 4449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:51.559103 4449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:44863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:51.560483 4449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:51.561806 4449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:51.574787 4455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:51.574753 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:51.575415 4454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:51.575759 4449 server_base.cc:1034] running on GCE node
I20260926 21:28:51.576634 4449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:51.578284 4449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:51.579578 4449 hybrid_clock.cc:648] HybridClock initialized: now 1790458131579521 us; error 56 us; skew 500 ppm
I20260926 21:28:51.580019 4449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:51 dist-test-slave-4d5t krb5kdc[4436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458131, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:51.590128 4449 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:51.592118 4449 webserver.cc:460] Webserver started at http://127.14.142.62:39385/ using document root <none> and password file <none>
I20260926 21:28:51.593006 4449 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:51.593169 4449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:51.593506 4449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:51.596951 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "5d2c2de28ac04cac81813eb8a970d742"
format_stamp: "Formatted at 2026-09-26 21:28:51 on dist-test-slave-4d5t"
server_key: "277038b6932b70331123e55054c3bd35"
server_key_iv: "674b382422da5d679d8ddfb5e516524e"
server_key_version: "encryptionkey@0"
I20260926 21:28:51.598038 4449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "5d2c2de28ac04cac81813eb8a970d742"
format_stamp: "Formatted at 2026-09-26 21:28:51 on dist-test-slave-4d5t"
server_key: "277038b6932b70331123e55054c3bd35"
server_key_iv: "674b382422da5d679d8ddfb5e516524e"
server_key_version: "encryptionkey@0"
I20260926 21:28:51.604720 4449 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260926 21:28:51.608785 4464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:51.610499 4449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 21:28:51.610806 4449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "5d2c2de28ac04cac81813eb8a970d742"
format_stamp: "Formatted at 2026-09-26 21:28:51 on dist-test-slave-4d5t"
server_key: "277038b6932b70331123e55054c3bd35"
server_key_iv: "674b382422da5d679d8ddfb5e516524e"
server_key_version: "encryptionkey@0"
I20260926 21:28:51.611047 4449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:51.617491 4449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:51.622195 4449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:51.622534 4449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:51.646517 4449 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:44863
I20260926 21:28:51.646557 4515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:44863 every 8 connection(s)
I20260926 21:28:51.648113 4449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:51.652238 4516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:51.658401 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4449
I20260926 21:28:51.658599 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:51.659139 14904 external_mini_cluster.cc:1428] Setting key 0d5a129cb9015a193b09cf7a7ee9971f
I20260926 21:28:51.663882 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742: Bootstrap starting.
I20260926 21:28:51.667379 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:51.661113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46561 (local address 127.14.142.62:44863)
0926 21:28:51.662073 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:51.662086 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:51.662130 (+ 44us) server_negotiation.cc:407] Connection header received
0926 21:28:51.663843 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:51.663869 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:51.664278 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:51.664673 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:51.665763 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:51.665770 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:51.665782 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:51.665809 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:51.665870 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:51.665895 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:51.666106 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:51.666218 (+ 112us) server_negotiation.cc:299] Negotiation successful
0926 21:28:51.666435 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":480,"threads_started":1}
I20260926 21:28:51.667564 4516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:51.668815 4516 log.cc:824] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:51.672394 4516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742: No bootstrap required, opened a new log
I20260926 21:28:51.672447 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:51.681396 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:51.678570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54565 (local address 127.14.142.62:44863)
0926 21:28:51.678786 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:51.678793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:51.679172 (+ 379us) server_negotiation.cc:407] Connection header received
0926 21:28:51.679324 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:51.679331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:51.679403 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:51.679575 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:51.680612 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:51.680618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:51.680623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:51.680643 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:51.680699 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:51.680721 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:51.680880 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:51.680995 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 21:28:51.681055 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 21:28:51.684108 4516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2c2de28ac04cac81813eb8a970d742" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44863 } }
I20260926 21:28:51.684535 4516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:51.684686 4516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d2c2de28ac04cac81813eb8a970d742, State: Initialized, Role: FOLLOWER
I20260926 21:28:51.685359 4516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2c2de28ac04cac81813eb8a970d742" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44863 } }
I20260926 21:28:51.685652 4516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:51.685806 4516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:51.686000 4516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:51.688436 4516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2c2de28ac04cac81813eb8a970d742" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44863 } }
I20260926 21:28:51.689033 4516 leader_election.cc:302] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d2c2de28ac04cac81813eb8a970d742; no voters:
I20260926 21:28:51.689847 4516 leader_election.cc:288] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:51.689985 4526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:51.690665 4526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [term 1 LEADER]: Becoming Leader. State: Replica: 5d2c2de28ac04cac81813eb8a970d742, State: Running, Role: LEADER
I20260926 21:28:51.691370 4526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2c2de28ac04cac81813eb8a970d742" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44863 } }
I20260926 21:28:51.692432 4516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:51.694875 4527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d2c2de28ac04cac81813eb8a970d742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2c2de28ac04cac81813eb8a970d742" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44863 } } }
I20260926 21:28:51.694844 4528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d2c2de28ac04cac81813eb8a970d742. Latest consensus state: current_term: 1 leader_uuid: "5d2c2de28ac04cac81813eb8a970d742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d2c2de28ac04cac81813eb8a970d742" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 44863 } } }
I20260926 21:28:51.695461 4527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:51.695494 4528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:51.696657 4532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:51.703940 4532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:51.714349 4532 catalog_manager.cc:1347] Generated new cluster ID: ff6a0e5360904525b720eb8619706a86
I20260926 21:28:51.714493 4532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:51.730376 4532 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:51.730837 4532 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:51.740679 4532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d2c2de28ac04cac81813eb8a970d742: Generated new TSK 0
I20260926 21:28:51.741572 4532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:51.748906 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4449
W20260926 21:28:51.750186 4511 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46561
W20260926 21:28:51.751050 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4449: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:51.751226 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4449
2026-09-26T21:28:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2017 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:51 dist-test-slave-4d5t krb5kdc[4547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:51 dist-test-slave-4d5t krb5kdc[4547](info): set up 2 sockets
Sep 26 21:28:51 dist-test-slave-4d5t krb5kdc[4547](info): commencing operation
krb5kdc: starting...
W20260926 21:28:52.902060 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:53 dist-test-slave-4d5t krb5kdc[4547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:53.208205 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38611
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:42065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:38611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:53.563035 4560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:53.563370 4560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:53.563565 4560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:53.571201 4560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:53.571336 4560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:53.571421 4560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:53.571544 4560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:53.571607 4560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:53.582039 4560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:42065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:38611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:38611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:53.583425 4560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:53.584836 4560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:53.597728 4565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:53.597992 4568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:53.598024 4566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:53.599443 4560 server_base.cc:1034] running on GCE node
I20260926 21:28:53.600843 4560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:53.602923 4560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:53.604400 4560 hybrid_clock.cc:648] HybridClock initialized: now 1790458133604296 us; error 79 us; skew 500 ppm
I20260926 21:28:53.605031 4560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:53 dist-test-slave-4d5t krb5kdc[4547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458133, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:53.609184 4560 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:53.611385 4560 webserver.cc:460] Webserver started at http://127.14.142.62:42697/ using document root <none> and password file <none>
I20260926 21:28:53.612175 4560 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:53.612332 4560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:53.612748 4560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:53.615984 4560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "ac594deab4aa4283aa5302e58f999e7b"
format_stamp: "Formatted at 2026-09-26 21:28:53 on dist-test-slave-4d5t"
server_key: "8985ab74837530f6dd778cd9ef6f461e"
server_key_iv: "bac9e2e4c52b9a21cb00509332c0876c"
server_key_version: "encryptionkey@0"
I20260926 21:28:53.616904 4560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "ac594deab4aa4283aa5302e58f999e7b"
format_stamp: "Formatted at 2026-09-26 21:28:53 on dist-test-slave-4d5t"
server_key: "8985ab74837530f6dd778cd9ef6f461e"
server_key_iv: "bac9e2e4c52b9a21cb00509332c0876c"
server_key_version: "encryptionkey@0"
I20260926 21:28:53.623206 4560 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:28:53.627521 4575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:53.629124 4560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 21:28:53.629285 4560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "ac594deab4aa4283aa5302e58f999e7b"
format_stamp: "Formatted at 2026-09-26 21:28:53 on dist-test-slave-4d5t"
server_key: "8985ab74837530f6dd778cd9ef6f461e"
server_key_iv: "bac9e2e4c52b9a21cb00509332c0876c"
server_key_version: "encryptionkey@0"
I20260926 21:28:53.629483 4560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:53.650888 4560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:53.655681 4560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:53.655956 4560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:53.678673 4560 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:38611
I20260926 21:28:53.678733 4626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:38611 every 8 connection(s)
I20260926 21:28:53.680416 4560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:53.684445 4627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:53.688596 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4560
I20260926 21:28:53.688789 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:53.689329 14904 external_mini_cluster.cc:1428] Setting key a3af815ea95f1adcf75da6f3c5456c34
I20260926 21:28:53.697453 4627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b: Bootstrap starting.
I20260926 21:28:53.700665 4627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:53.701710 4627 log.cc:824] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b: Log is configured to *not* fsync() on all Append() calls
Sep 26 21:28:53 dist-test-slave-4d5t krb5kdc[4547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:28:53.704676 4627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b: No bootstrap required, opened a new log
I20260926 21:28:53.713243 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:53.691352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36145 (local address 127.14.142.62:38611)
0926 21:28:53.692179 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:53.692191 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:53.692233 (+ 42us) server_negotiation.cc:407] Connection header received
0926 21:28:53.693950 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:53.693984 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:53.694409 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:53.694814 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:53.696276 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:53.697741 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:53.699026 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:53.699688 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:53.699889 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:53.704112 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:53.704120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:53.704134 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:53.704171 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:53.708152 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:53.709208 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:53.709216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:53.709223 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:53.709486 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:53.709920 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:53.709926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:53.709930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:53.710608 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:53.710990 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:53.711524 (+ 534us) server_negotiation.cc:299] Negotiation successful
0926 21:28:53.712025 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":393,"threads_started":1}
I20260926 21:28:53.713367 4627 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac594deab4aa4283aa5302e58f999e7b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38611 } }
I20260926 21:28:53.713876 4627 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:53.714004 4627 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac594deab4aa4283aa5302e58f999e7b, State: Initialized, Role: FOLLOWER
I20260926 21:28:53.714704 4627 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac594deab4aa4283aa5302e58f999e7b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38611 } }
I20260926 21:28:53.714970 4627 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:53.715127 4627 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:53.715308 4627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:53.717599 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:53.718114 4627 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac594deab4aa4283aa5302e58f999e7b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38611 } }
I20260926 21:28:53.718607 4627 leader_election.cc:302] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac594deab4aa4283aa5302e58f999e7b; no voters:
I20260926 21:28:53.719221 4627 leader_election.cc:288] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:53.719416 4632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:53.720671 4632 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [term 1 LEADER]: Becoming Leader. State: Replica: ac594deab4aa4283aa5302e58f999e7b, State: Running, Role: LEADER
I20260926 21:28:53.721396 4627 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:53.721566 4632 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac594deab4aa4283aa5302e58f999e7b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38611 } }
I20260926 21:28:53.729977 4633 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac594deab4aa4283aa5302e58f999e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac594deab4aa4283aa5302e58f999e7b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38611 } } }
I20260926 21:28:53.730451 4633 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:53.731077 4634 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac594deab4aa4283aa5302e58f999e7b. Latest consensus state: current_term: 1 leader_uuid: "ac594deab4aa4283aa5302e58f999e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac594deab4aa4283aa5302e58f999e7b" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 38611 } } }
I20260926 21:28:53.731393 4634 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:53.739992 4643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:53.747843 4643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:53.771634 4643 catalog_manager.cc:1347] Generated new cluster ID: 81f438c1477b45b492b752bacb93cc4f
I20260926 21:28:53.771770 4643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:53.779515 4630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:53.751982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42765 (local address 127.14.142.62:38611)
0926 21:28:53.752241 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:53.752251 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:53.758075 (+ 5824us) server_negotiation.cc:407] Connection header received
0926 21:28:53.758403 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:53.758410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:53.758525 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:53.758633 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:53.761033 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:53.762299 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:53.765273 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:53.766351 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:53.766499 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:53.770677 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:53.770685 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:53.770691 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:28:53.770742 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:53.775027 (+ 4285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:53.776037 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:53.776044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:53.776048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:53.776199 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:28:53.776750 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:28:53.776756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:28:53.776760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:28:53.777235 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:53.777522 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:53.778806 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0926 21:28:53.779064 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260926 21:28:53.810078 4643 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:53.811637 4643 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:53.822984 4643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac594deab4aa4283aa5302e58f999e7b: Generated new TSK 0
I20260926 21:28:53.824280 4643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:53.834050 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4560
I20260926 21:28:53.835507 4622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:28:53.935428 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4560
2026-09-26T21:28:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2184 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:54 dist-test-slave-4d5t krb5kdc[4660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:54 dist-test-slave-4d5t krb5kdc[4660](info): set up 2 sockets
Sep 26 21:28:54 dist-test-slave-4d5t krb5kdc[4660](info): commencing operation
krb5kdc: starting...
W20260926 21:28:55.118408 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:55 dist-test-slave-4d5t krb5kdc[4660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:55.412860 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:45753
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:40493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:45753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:55.774309 4673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:55.774631 4673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:55.774775 4673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:55.782259 4673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:55.782402 4673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:55.782510 4673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:55.782610 4673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:55.782653 4673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:55.793449 4673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:40493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:45753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:45753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:55.794930 4673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:55.796295 4673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:55.809999 4678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:55.810001 4679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:55.809952 4681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:55.810848 4673 server_base.cc:1034] running on GCE node
I20260926 21:28:55.811625 4673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:55.813143 4673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:55.814386 4673 hybrid_clock.cc:648] HybridClock initialized: now 1790458135814304 us; error 60 us; skew 500 ppm
I20260926 21:28:55.814734 4673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:55 dist-test-slave-4d5t krb5kdc[4660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458135, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:55.818154 4673 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:55.819885 4673 webserver.cc:460] Webserver started at http://127.14.142.62:45767/ using document root <none> and password file <none>
I20260926 21:28:55.820643 4673 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:55.820775 4673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:55.821098 4673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:55.824208 4673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "1cc7ce3156e24c949004f6d9644c46ed"
format_stamp: "Formatted at 2026-09-26 21:28:55 on dist-test-slave-4d5t"
server_key: "14b3f3f5184a37580219df76c8ad20eb"
server_key_iv: "d6f893029f6d2c52fac70ab15c3df181"
server_key_version: "encryptionkey@0"
I20260926 21:28:55.825039 4673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "1cc7ce3156e24c949004f6d9644c46ed"
format_stamp: "Formatted at 2026-09-26 21:28:55 on dist-test-slave-4d5t"
server_key: "14b3f3f5184a37580219df76c8ad20eb"
server_key_iv: "d6f893029f6d2c52fac70ab15c3df181"
server_key_version: "encryptionkey@0"
I20260926 21:28:55.830365 4673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 21:28:55.834421 4688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:55.835853 4673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 21:28:55.836045 4673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "1cc7ce3156e24c949004f6d9644c46ed"
format_stamp: "Formatted at 2026-09-26 21:28:55 on dist-test-slave-4d5t"
server_key: "14b3f3f5184a37580219df76c8ad20eb"
server_key_iv: "d6f893029f6d2c52fac70ab15c3df181"
server_key_version: "encryptionkey@0"
I20260926 21:28:55.836241 4673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:55.864754 4673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:55.869421 4673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:55.869750 4673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:55.890414 4673 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:45753
I20260926 21:28:55.890439 4739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:45753 every 8 connection(s)
I20260926 21:28:55.891832 4673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:55.895985 4740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:55.896420 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4673
I20260926 21:28:55.896647 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:55.897190 14904 external_mini_cluster.cc:1428] Setting key 3e99d9df32601d722833f55ce2870ac1
I20260926 21:28:55.910559 4740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed: Bootstrap starting.
I20260926 21:28:55.913306 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:55.899223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53067 (local address 127.14.142.62:45753)
0926 21:28:55.900215 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:55.900227 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:55.900274 (+ 47us) server_negotiation.cc:407] Connection header received
0926 21:28:55.902673 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:55.902703 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:55.903174 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:55.903545 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:55.904154 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:55.905639 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:55.907693 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:55.908867 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:55.909138 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:55.910498 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:55.910508 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:55.910525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:55.911210 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:55.911289 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:55.911319 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:55.911414 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:55.911719 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 21:28:55.912049 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":484,"threads_started":1}
I20260926 21:28:55.914234 4740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:55.915341 4740 log.cc:824] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:55.917328 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:55.918699 4740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed: No bootstrap required, opened a new log
I20260926 21:28:55.929368 4740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc7ce3156e24c949004f6d9644c46ed" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45753 } }
I20260926 21:28:55.929960 4740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:55.930107 4740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc7ce3156e24c949004f6d9644c46ed, State: Initialized, Role: FOLLOWER
I20260926 21:28:55.930604 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:55.922525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60281 (local address 127.14.142.62:45753)
0926 21:28:55.922721 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:55.922728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:55.923244 (+ 516us) server_negotiation.cc:407] Connection header received
0926 21:28:55.923415 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:55.923422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:55.923507 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:55.923627 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:55.924547 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:55.925724 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:55.927219 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:28:55.927979 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:28:55.928127 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:28:55.929566 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:55.929573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:55.929577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:55.929601 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:55.929656 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:55.929679 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:55.929804 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:55.930102 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 21:28:55.930201 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 21:28:55.931059 4740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc7ce3156e24c949004f6d9644c46ed" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45753 } }
I20260926 21:28:55.931437 4740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:55.931806 4740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:55.932044 4740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:55.935897 4740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc7ce3156e24c949004f6d9644c46ed" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45753 } }
I20260926 21:28:55.936947 4740 leader_election.cc:302] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cc7ce3156e24c949004f6d9644c46ed; no voters:
I20260926 21:28:55.937772 4740 leader_election.cc:288] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:55.937985 4750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:55.939945 4750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [term 1 LEADER]: Becoming Leader. State: Replica: 1cc7ce3156e24c949004f6d9644c46ed, State: Running, Role: LEADER
I20260926 21:28:55.941102 4740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:55.940953 4750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc7ce3156e24c949004f6d9644c46ed" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45753 } }
I20260926 21:28:55.944635 4752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cc7ce3156e24c949004f6d9644c46ed. Latest consensus state: current_term: 1 leader_uuid: "1cc7ce3156e24c949004f6d9644c46ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc7ce3156e24c949004f6d9644c46ed" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45753 } } }
I20260926 21:28:55.945106 4752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:55.946238 4751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cc7ce3156e24c949004f6d9644c46ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc7ce3156e24c949004f6d9644c46ed" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45753 } } }
I20260926 21:28:55.946928 4751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:55.952757 4759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:55.957520 4759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:55.968246 4759 catalog_manager.cc:1347] Generated new cluster ID: 05ff2116a28644519149a3cec6cad280
I20260926 21:28:55.968380 4759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:55.978655 4759 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:55.980053 4759 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:55.986788 4759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cc7ce3156e24c949004f6d9644c46ed: Generated new TSK 0
I20260926 21:28:55.987927 4759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:55.995322 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4673
W20260926 21:28:55.996814 4735 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53067
W20260926 21:28:55.997684 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4673: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:55.997895 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4673
2026-09-26T21:28:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:56 dist-test-slave-4d5t krb5kdc[4771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:56 dist-test-slave-4d5t krb5kdc[4771](info): set up 2 sockets
Sep 26 21:28:56 dist-test-slave-4d5t krb5kdc[4771](info): commencing operation
krb5kdc: starting...
W20260926 21:28:57.225636 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:57 dist-test-slave-4d5t krb5kdc[4771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:57.532822 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:42515
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:36537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:57.888899 4784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:57.889242 4784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:57.889400 4784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:57.897159 4784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:57.897291 4784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:57.897372 4784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:57.897503 4784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:57.897586 4784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:28:57.908003 4784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:36537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:28:57.909415 4784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:28:57.910760 4784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:28:57.922894 4789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:57.923283 4792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:28:57.923313 4790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:28:57.923821 4784 server_base.cc:1034] running on GCE node
I20260926 21:28:57.924733 4784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:28:57.926256 4784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:28:57.927518 4784 hybrid_clock.cc:648] HybridClock initialized: now 1790458137927469 us; error 47 us; skew 500 ppm
I20260926 21:28:57.927929 4784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:28:57 dist-test-slave-4d5t krb5kdc[4771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458137, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:28:57.936319 4784 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:28:57.938225 4784 webserver.cc:460] Webserver started at http://127.14.142.62:39533/ using document root <none> and password file <none>
I20260926 21:28:57.938985 4784 fs_manager.cc:359] Metadata directory not provided
I20260926 21:28:57.939144 4784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:28:57.939468 4784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:28:57.942796 4784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "e6bac3a7c93242b98d70a459f66c77e8"
format_stamp: "Formatted at 2026-09-26 21:28:57 on dist-test-slave-4d5t"
server_key: "c5d29a6194f744de118d463db5ac5f32"
server_key_iv: "70a3b8d68dd6ac64ee6330656d32e549"
server_key_version: "encryptionkey@0"
I20260926 21:28:57.943636 4784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "e6bac3a7c93242b98d70a459f66c77e8"
format_stamp: "Formatted at 2026-09-26 21:28:57 on dist-test-slave-4d5t"
server_key: "c5d29a6194f744de118d463db5ac5f32"
server_key_iv: "70a3b8d68dd6ac64ee6330656d32e549"
server_key_version: "encryptionkey@0"
I20260926 21:28:57.949489 4784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 21:28:57.953154 4799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:28:57.954892 4784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 21:28:57.955093 4784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "e6bac3a7c93242b98d70a459f66c77e8"
format_stamp: "Formatted at 2026-09-26 21:28:57 on dist-test-slave-4d5t"
server_key: "c5d29a6194f744de118d463db5ac5f32"
server_key_iv: "70a3b8d68dd6ac64ee6330656d32e549"
server_key_version: "encryptionkey@0"
I20260926 21:28:57.955330 4784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:28:57.961365 4784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:28:57.965814 4784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:28:57.966145 4784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:28:57.987890 4784 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:42515
I20260926 21:28:57.987959 4850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:42515 every 8 connection(s)
I20260926 21:28:57.989539 4784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:28:57.993750 4851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:28:57.995370 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4784
I20260926 21:28:57.995590 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:28:57.996189 14904 external_mini_cluster.cc:1428] Setting key eff8b04bbedd6ef43ba76c179f867518
I20260926 21:28:58.004411 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:57.998294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44741 (local address 127.14.142.62:42515)
0926 21:28:57.999191 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:57.999206 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:57.999253 (+ 47us) server_negotiation.cc:407] Connection header received
0926 21:28:58.000984 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:58.001009 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:58.001369 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:58.001779 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:58.002812 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:58.002818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:58.002830 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:58.002844 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:58.002910 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:58.002930 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:58.003166 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:58.003298 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 21:28:58.003480 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":390,"threads_started":1}
I20260926 21:28:58.007855 4851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8: Bootstrap starting.
I20260926 21:28:58.008060 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:28:58.011152 4851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8: Neither blocks nor log segments found. Creating new log.
I20260926 21:28:58.012277 4851 log.cc:824] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8: Log is configured to *not* fsync() on all Append() calls
I20260926 21:28:58.015793 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:28:58.013099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47921 (local address 127.14.142.62:42515)
0926 21:28:58.013331 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 21:28:58.013338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:28:58.013661 (+ 323us) server_negotiation.cc:407] Connection header received
0926 21:28:58.013988 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:28:58.013994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:28:58.014056 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:28:58.014188 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:28:58.015072 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:28:58.015078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:28:58.015084 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 21:28:58.015111 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:28:58.015164 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 21:28:58.015188 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:28:58.015320 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 21:28:58.015413 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 21:28:58.015485 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 21:28:58.016211 4851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8: No bootstrap required, opened a new log
I20260926 21:28:58.025157 4851 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bac3a7c93242b98d70a459f66c77e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42515 } }
I20260926 21:28:58.025636 4851 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 21:28:58.025774 4851 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6bac3a7c93242b98d70a459f66c77e8, State: Initialized, Role: FOLLOWER
I20260926 21:28:58.026517 4851 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bac3a7c93242b98d70a459f66c77e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42515 } }
I20260926 21:28:58.026846 4851 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 21:28:58.027048 4851 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 21:28:58.027253 4851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:28:58.029826 4851 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bac3a7c93242b98d70a459f66c77e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42515 } }
I20260926 21:28:58.030431 4851 leader_election.cc:302] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6bac3a7c93242b98d70a459f66c77e8; no voters:
I20260926 21:28:58.031131 4851 leader_election.cc:288] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 21:28:58.031302 4861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:28:58.032080 4861 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [term 1 LEADER]: Becoming Leader. State: Replica: e6bac3a7c93242b98d70a459f66c77e8, State: Running, Role: LEADER
I20260926 21:28:58.032845 4861 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bac3a7c93242b98d70a459f66c77e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42515 } }
I20260926 21:28:58.033674 4851 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:28:58.037498 4863 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6bac3a7c93242b98d70a459f66c77e8. Latest consensus state: current_term: 1 leader_uuid: "e6bac3a7c93242b98d70a459f66c77e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bac3a7c93242b98d70a459f66c77e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42515 } } }
I20260926 21:28:58.037429 4862 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6bac3a7c93242b98d70a459f66c77e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6bac3a7c93242b98d70a459f66c77e8" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 42515 } } }
I20260926 21:28:58.037976 4862 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:58.037972 4863 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8 [sys.catalog]: This master's current role is: LEADER
I20260926 21:28:58.041829 4871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:28:58.046536 4871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:28:58.055447 4871 catalog_manager.cc:1347] Generated new cluster ID: 0e671e890be749bb91fb670661fec7b8
I20260926 21:28:58.055576 4871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:28:58.090812 4871 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:28:58.091462 4871 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:28:58.102084 4871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6bac3a7c93242b98d70a459f66c77e8: Generated new TSK 0
I20260926 21:28:58.103271 4871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:28:58.112884 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4784
W20260926 21:28:58.114452 4846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44741
W20260926 21:28:58.115206 14904 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4784: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 21:28:58.115370 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4784
2026-09-26T21:28:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2117 ms)
[----------] 18 tests from AuthTokenIssuingTest (37598 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:28:58 dist-test-slave-4d5t krb5kdc[4882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:28:58 dist-test-slave-4d5t krb5kdc[4882](info): set up 2 sockets
Sep 26 21:28:58 dist-test-slave-4d5t krb5kdc[4882](info): commencing operation
krb5kdc: starting...
W20260926 21:28:59.301736 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:28:59 dist-test-slave-4d5t krb5kdc[4882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:28:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:28:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:28:59.616441 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:45909
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:45909,127.14.142.61:43363,127.14.142.60:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:28:59.981016 4895 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 21:28:59.981451 4895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:28:59.981588 4895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:28:59.981731 4895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:28:59.989487 4895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:28:59.989629 4895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:28:59.989746 4895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:28:59.989843 4895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:28:59.989961 4895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:29:00.000576 4895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:45909,127.14.142.61:43363,127.14.142.60:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:45909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:29:00.002025 4895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:29:00.003372 4895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:29:00.015728 4903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:00.015686 4900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:00.015769 4901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:29:00.017328 4895 server_base.cc:1034] running on GCE node
I20260926 21:29:00.018276 4895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:29:00.020084 4895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:29:00.021386 4895 hybrid_clock.cc:648] HybridClock initialized: now 1790458140021360 us; error 117 us; skew 500 ppm
I20260926 21:29:00.021778 4895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:29:00 dist-test-slave-4d5t krb5kdc[4882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:29:00.025322 4895 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:29:00.026973 4895 webserver.cc:460] Webserver started at http://127.14.142.62:37035/ using document root <none> and password file <none>
I20260926 21:29:00.027700 4895 fs_manager.cc:359] Metadata directory not provided
I20260926 21:29:00.027837 4895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:29:00.028177 4895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:29:00.031527 4895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "dd656a670d7546db985f4ae7ec2d8615"
format_stamp: "Formatted at 2026-09-26 21:29:00 on dist-test-slave-4d5t"
server_key: "576debb4f5a6c7ddabc20a787049d499"
server_key_iv: "917ef6723deb47784c9ff3adc56b00ed"
server_key_version: "encryptionkey@0"
I20260926 21:29:00.032539 4895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "dd656a670d7546db985f4ae7ec2d8615"
format_stamp: "Formatted at 2026-09-26 21:29:00 on dist-test-slave-4d5t"
server_key: "576debb4f5a6c7ddabc20a787049d499"
server_key_iv: "917ef6723deb47784c9ff3adc56b00ed"
server_key_version: "encryptionkey@0"
I20260926 21:29:00.039206 4895 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:29:00.043346 4910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:29:00.045142 4895 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260926 21:29:00.045339 4895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "dd656a670d7546db985f4ae7ec2d8615"
format_stamp: "Formatted at 2026-09-26 21:29:00 on dist-test-slave-4d5t"
server_key: "576debb4f5a6c7ddabc20a787049d499"
server_key_iv: "917ef6723deb47784c9ff3adc56b00ed"
server_key_version: "encryptionkey@0"
I20260926 21:29:00.045595 4895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:29:00.077083 4895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:29:00.082437 4895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:29:00.082793 4895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:29:00.107182 4895 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:45909
I20260926 21:29:00.107200 4962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:45909 every 8 connection(s)
I20260926 21:29:00.108702 4895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:29:00.113952 4963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:29:00.118969 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4895
I20260926 21:29:00.119238 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:29:00.120148 14904 external_mini_cluster.cc:1428] Setting key 7d47c19edf8cedf781e820525a63feb3
I20260926 21:29:00.123329 4963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:29:00 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.142.61@KRBTEST.COM; defaulting to no policy
I20260926 21:29:00.153596 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.127963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:45909 (local address 127.0.0.1:43598)
0926 21:29:00.129401 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.129480 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:00.132703 (+ 3223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:00.133786 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:00.133796 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:00.134474 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:00.134816 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.134833 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.137316 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.137321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.138615 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.138628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.140020 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.140026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.140159 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.140945 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:00.140970 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:00.143173 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:00.148282 (+ 5109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.148302 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.148318 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.149276 (+ 958us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.149916 (+ 640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.149922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.149926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.150211 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.151379 (+ 1168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:00.151385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:00.151707 (+ 322us) client_negotiation.cc:769] Sending connection context
0926 21:29:00.151944 (+ 237us) client_negotiation.cc:240] Negotiation successful
0926 21:29:00.152373 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":565,"threads_started":1}
I20260926 21:29:00.153808 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.127963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.14.142.62:45909)
0926 21:29:00.130331 (+ 2368us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:00.130339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:00.130392 (+ 53us) server_negotiation.cc:407] Connection header received
0926 21:29:00.133208 (+ 2816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:00.133215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:00.133521 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:00.133694 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:00.135051 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.137086 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.138779 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.139756 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.139987 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.143452 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:00.143469 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:00.143492 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:00.143538 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:00.148003 (+ 4465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.149471 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.149485 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.149492 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.149740 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.150357 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.150365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.150371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.151168 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:00.151394 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:00.152158 (+ 764us) server_negotiation.cc:299] Negotiation successful
0926 21:29:00.152625 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":1996,"spinlock_wait_cycles":34560,"thread_start_us":1744,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.61@KRBTEST.COM" created.
I20260926 21:29:00.157145 4963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:00.158303 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.157882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44748)
0926 21:29:00.158059 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.158172 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 21:29:00.158586 4911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
I20260926 21:29:00.160205 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.159834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44752)
0926 21:29:00.159987 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.160069 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 21:29:00.160394 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:43363: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab.
Entry for principal kudu/127.14.142.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab.
I20260926 21:29:00.192745 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:43363
--webserver_interface=127.14.142.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab
--principal=kudu/127.14.142.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:45909,127.14.142.61:43363,127.14.142.60:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:29:00.209738 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } attempt: 1
I20260926 21:29:00.210868 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.210431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44762)
0926 21:29:00.210628 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.210737 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 21:29:00.211789 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.211461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44772)
0926 21:29:00.211599 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.211669 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 21:29:00.211980 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:43363: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
I20260926 21:29:00.280293 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } attempt: 2
I20260926 21:29:00.281668 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.281080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44788)
0926 21:29:00.281337 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.281456 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 21:29:00.282764 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.282354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44794)
0926 21:29:00.282501 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.282594 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 21:29:00.283020 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:43363: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
I20260926 21:29:00.374333 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } attempt: 3
I20260926 21:29:00.375522 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.375083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44796)
0926 21:29:00.375283 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.375381 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 21:29:00.376542 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.376193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44810)
0926 21:29:00.376342 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.376407 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 21:29:00.376960 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:43363: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
I20260926 21:29:00.520254 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } attempt: 4
I20260926 21:29:00.521653 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.521075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44820)
0926 21:29:00.521426 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.521514 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260926 21:29:00.522873 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.522453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44824)
0926 21:29:00.522636 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.522741 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260926 21:29:00.523125 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:43363: Network error: Client connection negotiation failed: client connection to 127.14.142.61:43363: connect: Connection refused (error 111)
W20260926 21:29:00.595973 4970 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 21:29:00.596350 4970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:29:00.596498 4970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:29:00.596681 4970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:29:00.604480 4970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:29:00.604682 4970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:29:00.604802 4970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:29:00.604892 4970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:29:00.604981 4970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:29:00.617234 4970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:45909,127.14.142.61:43363,127.14.142.60:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab
--principal=kudu/127.14.142.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:43363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.142.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:29:00.618698 4970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:29:00.620075 4970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:29:00.634969 4975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:00.635396 4978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:00.635552 4976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:29:00.636741 4970 server_base.cc:1034] running on GCE node
I20260926 21:29:00.637707 4970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:29:00.639204 4970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:29:00.640475 4970 hybrid_clock.cc:648] HybridClock initialized: now 1790458140640393 us; error 63 us; skew 500 ppm
I20260926 21:29:00.640862 4970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:29:00 dist-test-slave-4d5t krb5kdc[4882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:29:00.644210 4970 init.cc:377] Logged in from keytab as kudu/127.14.142.61@KRBTEST.COM (short username kudu)
I20260926 21:29:00.645951 4970 webserver.cc:460] Webserver started at http://127.14.142.61:42623/ using document root <none> and password file <none>
I20260926 21:29:00.646745 4970 fs_manager.cc:359] Metadata directory not provided
I20260926 21:29:00.646895 4970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:29:00.647251 4970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:29:00.651027 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data/instance:
uuid: "23599be46ee745bab7e77db8faaab722"
format_stamp: "Formatted at 2026-09-26 21:29:00 on dist-test-slave-4d5t"
server_key: "47cebb2729bd9ecbfbf9b0ca7defebc4"
server_key_iv: "53786fc5ceadc935ca809192f684e863"
server_key_version: "encryptionkey@0"
I20260926 21:29:00.651955 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal/instance:
uuid: "23599be46ee745bab7e77db8faaab722"
format_stamp: "Formatted at 2026-09-26 21:29:00 on dist-test-slave-4d5t"
server_key: "47cebb2729bd9ecbfbf9b0ca7defebc4"
server_key_iv: "53786fc5ceadc935ca809192f684e863"
server_key_version: "encryptionkey@0"
I20260926 21:29:00.658649 4970 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260926 21:29:00.663384 4985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:29:00.665195 4970 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 21:29:00.665426 4970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal
uuid: "23599be46ee745bab7e77db8faaab722"
format_stamp: "Formatted at 2026-09-26 21:29:00 on dist-test-slave-4d5t"
server_key: "47cebb2729bd9ecbfbf9b0ca7defebc4"
server_key_iv: "53786fc5ceadc935ca809192f684e863"
server_key_version: "encryptionkey@0"
I20260926 21:29:00.665710 4970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:29:00.705188 4970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:29:00.710584 4970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:29:00.710947 4970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:29:00.734918 4970 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.61:43363
I20260926 21:29:00.734972 5036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.61:43363 every 8 connection(s)
I20260926 21:29:00.736390 4970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
I20260926 21:29:00.740341 5037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:29:00.745646 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 4970
I20260926 21:29:00.745862 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-1/wal/instance
I20260926 21:29:00.746394 14904 external_mini_cluster.cc:1428] Setting key 6de4910d0397b4e1d1d39ae057c5c1ee
I20260926 21:29:00.748303 5037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:29:00 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:29:00.775868 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.752135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.14.142.62:45909)
0926 21:29:00.752753 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:00.752762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:00.753890 (+ 1128us) server_negotiation.cc:407] Connection header received
0926 21:29:00.755894 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:00.755900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:00.755999 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:00.756121 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:00.757920 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.759189 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.763065 (+ 3876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.763623 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.763791 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.768996 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:00.769002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:00.769006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:00.769061 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:00.772430 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.773334 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.773339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.773341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.773470 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.773997 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.774000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.774003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.774313 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:00.774473 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:00.775238 (+ 765us) server_negotiation.cc:299] Negotiation successful
0926 21:29:00.775484 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":301,"threads_started":1}
I20260926 21:29:00.777029 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.752837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:45909 (local address 127.0.0.1:43614)
0926 21:29:00.753676 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.753739 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:00.755588 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:00.756379 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:00.756388 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:00.756998 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:00.757638 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.757657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.759375 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.759384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.762717 (+ 3333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.762744 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.764013 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.764028 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.764463 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.765450 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:00.765491 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:00.768666 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:00.772644 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.772655 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.772675 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.773175 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.773587 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.773592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.773598 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.773862 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.774445 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:00.774454 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:00.774865 (+ 411us) client_negotiation.cc:769] Sending connection context
0926 21:29:00.775132 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 21:29:00.775690 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":372,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.60@KRBTEST.COM; defaulting to no policy
I20260926 21:29:00.779326 5037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.60@KRBTEST.COM" created.
Sep 26 21:29:00 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:29:00.798476 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.780292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44834)
0926 21:29:00.780494 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.780519 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:00.780784 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:00.781536 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:00.781539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:00.782081 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:00.782262 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.782273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.783753 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.783762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.785346 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.785363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.786275 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.786283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.786520 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.787171 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:00.787198 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:00.790021 (+ 2823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:00.794993 (+ 4972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.794997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.795001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.795422 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.796253 (+ 831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.796256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.796259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.796424 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.797633 (+ 1209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:00.797637 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:00.797803 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 21:29:00.797933 (+ 130us) client_negotiation.cc:240] Negotiation successful
0926 21:29:00.798179 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 21:29:00.799398 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.780395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.14.142.61:43363)
0926 21:29:00.781074 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:00.781081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:00.781108 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:29:00.781159 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:00.781165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:00.781367 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:00.781504 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:00.782500 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.783592 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.785507 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.786099 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.786217 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.790283 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:00.790299 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:00.790315 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:00.790348 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:00.794750 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.795761 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.795778 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.795785 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.796067 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.796645 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.796653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.796659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.797433 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:00.797776 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:00.798185 (+ 409us) server_negotiation.cc:299] Negotiation successful
0926 21:29:00.798609 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":386,"threads_started":1}
I20260926 21:29:00.802055 5037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:00.803692 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.802997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45044)
0926 21:29:00.803287 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.803489 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260926 21:29:00.804173 4989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:00.806290 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.805891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45060)
0926 21:29:00.806064 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.806160 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 21:29:00.806564 5037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:00.822468 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab.
Entry for principal kudu/127.14.142.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab.
I20260926 21:29:00.825271 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:36139
--webserver_interface=127.14.142.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab
--principal=kudu/127.14.142.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:45909,127.14.142.61:43363,127.14.142.60:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 21:29:00 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:29:00.841455 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.823546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44838)
0926 21:29:00.823865 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.823889 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:00.824185 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:00.824795 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:00.824799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:00.825291 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:00.825653 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.825664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.827339 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.827346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.828686 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:00.828702 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.829876 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.829881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:00.830100 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.830923 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:00.830952 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:00.833805 (+ 2853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:00.838554 (+ 4749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.838662 (+ 108us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.838667 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.839150 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.839650 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:00.839655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:00.839659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:00.839797 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:00.840349 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:00.840354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:00.840522 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 21:29:00.840756 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 21:29:00.841080 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 21:29:00.841455 5042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.823854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.14.142.61:43363)
0926 21:29:00.824110 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:00.824119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:00.824146 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:29:00.824432 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:00.824440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:00.824578 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:00.824700 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:00.825836 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.827115 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.828867 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:00.829644 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:00.829815 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:00.834074 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:00.834084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:00.834092 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:00.834146 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:00.838316 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.839356 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.839363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.839373 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.839516 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:00.839909 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:00.839915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:00.839919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:00.840222 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:00.840393 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:00.840858 (+ 465us) server_negotiation.cc:299] Negotiation successful
0926 21:29:00.841083 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:29:00.842729 4963 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:00.844079 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.843493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45070)
0926 21:29:00.843724 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.843842 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 21:29:00.845221 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.844717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45074)
0926 21:29:00.844906 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.845044 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 21:29:00.845584 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:00.855919 5037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 1
I20260926 21:29:00.857290 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.856770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45082)
0926 21:29:00.856994 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.857124 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260926 21:29:00.858522 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.858142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45098)
0926 21:29:00.858318 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.858403 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 21:29:00.858847 5037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:00.896862 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 1
I20260926 21:29:00.898170 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.897606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45106)
0926 21:29:00.897841 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.897955 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 21:29:00.899515 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.899112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45116)
0926 21:29:00.899276 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.899366 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 21:29:00.899845 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:00.927095 5037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 2
I20260926 21:29:00.928484 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.927936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45118)
0926 21:29:00.928190 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.928293 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 21:29:00.929724 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.929346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45128)
0926 21:29:00.929521 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.929598 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 21:29:00.929986 5037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:00.968219 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 2
I20260926 21:29:00.969966 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.969293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45144)
0926 21:29:00.969655 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.969766 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
I20260926 21:29:00.971329 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:00.970878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45156)
0926 21:29:00.971074 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:00.971192 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260926 21:29:00.971762 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:01.021256 5037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 3
I20260926 21:29:01.022424 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.021967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45160)
0926 21:29:01.022166 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.022261 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 21:29:01.023483 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.023065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45168)
0926 21:29:01.023296 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.023374 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260926 21:29:01.023769 5037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:01.078269 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 3
I20260926 21:29:01.080010 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.079376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45180)
0926 21:29:01.079658 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.079795 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260926 21:29:01.081552 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.080972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45194)
0926 21:29:01.081353 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.081438 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
W20260926 21:29:01.081952 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
I20260926 21:29:01.167032 5037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 4
I20260926 21:29:01.168406 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.167898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45208)
0926 21:29:01.168136 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.168229 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260926 21:29:01.169555 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.169230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45212)
0926 21:29:01.169369 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.169439 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 21:29:01.169883 5037 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
W20260926 21:29:01.219504 5045 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 21:29:01.219955 5045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:29:01.220108 5045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:29:01.220259 5045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:29:01.228833 5045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:29:01.228997 5045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:29:01.229120 5045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:29:01.229231 5045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:29:01.229336 5045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:29:01.240787 5045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:45909,127.14.142.61:43363,127.14.142.60:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab
--principal=kudu/127.14.142.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:36139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.142.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:29:01.242246 5045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:29:01.243626 5045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:29:01.256246 5050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:01.256197 5051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:01.256273 5053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:29:01.257485 5045 server_base.cc:1034] running on GCE node
I20260926 21:29:01.258315 5045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:29:01.259200 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 4
I20260926 21:29:01.260021 5045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:29:01.260476 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.259918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45216)
0926 21:29:01.260155 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.260266 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260926 21:29:01.261323 5045 hybrid_clock.cc:648] HybridClock initialized: now 1790458141261234 us; error 49 us; skew 500 ppm
I20260926 21:29:01.261611 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.261185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45220)
0926 21:29:01.261370 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.261494 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 21:29:01.261749 5045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260926 21:29:01.261899 4963 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:36139: Network error: Client connection negotiation failed: client connection to 127.14.142.60:36139: connect: Connection refused (error 111)
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458141, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:29:01.265525 5045 init.cc:377] Logged in from keytab as kudu/127.14.142.60@KRBTEST.COM (short username kudu)
I20260926 21:29:01.267230 5045 webserver.cc:460] Webserver started at http://127.14.142.60:43665/ using document root <none> and password file <none>
I20260926 21:29:01.267987 5045 fs_manager.cc:359] Metadata directory not provided
I20260926 21:29:01.268419 5045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:29:01.269021 5045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:29:01.272526 5045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data/instance:
uuid: "3d5b9d277cd34add9f768603b74341a6"
format_stamp: "Formatted at 2026-09-26 21:29:01 on dist-test-slave-4d5t"
server_key: "e7e1bee618a3346bfa8696e4387fbe2e"
server_key_iv: "ef89152e592d3b02f629279a98cb6509"
server_key_version: "encryptionkey@0"
I20260926 21:29:01.273538 5045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal/instance:
uuid: "3d5b9d277cd34add9f768603b74341a6"
format_stamp: "Formatted at 2026-09-26 21:29:01 on dist-test-slave-4d5t"
server_key: "e7e1bee618a3346bfa8696e4387fbe2e"
server_key_iv: "ef89152e592d3b02f629279a98cb6509"
server_key_version: "encryptionkey@0"
I20260926 21:29:01.279959 5045 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 21:29:01.284705 5060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:29:01.286201 5045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:29:01.286422 5045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal
uuid: "3d5b9d277cd34add9f768603b74341a6"
format_stamp: "Formatted at 2026-09-26 21:29:01 on dist-test-slave-4d5t"
server_key: "e7e1bee618a3346bfa8696e4387fbe2e"
server_key_iv: "ef89152e592d3b02f629279a98cb6509"
server_key_version: "encryptionkey@0"
I20260926 21:29:01.286664 5045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:29:01.313434 5045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:29:01.318593 5045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:29:01.318921 5045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:29:01.341110 5045 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.60:36139
I20260926 21:29:01.341230 5111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.60:36139 every 8 connection(s)
I20260926 21:29:01.342494 5045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
I20260926 21:29:01.346793 5112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:29:01.351603 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 5045
I20260926 21:29:01.351792 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790457761955429-14904-0/minicluster-data/master-2/wal/instance
I20260926 21:29:01.352327 14904 external_mini_cluster.cc:1428] Setting key cdcb94cc32891e41d0acbcce12559404
I20260926 21:29:01.354777 5112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458141, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:29:01.376677 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.358932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.14.142.62:45909)
0926 21:29:01.359495 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.359501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.360029 (+ 528us) server_negotiation.cc:407] Connection header received
0926 21:29:01.360241 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.360247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.360330 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.360457 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.361688 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.362752 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.364486 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.365190 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.365335 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.368694 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.368699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.368702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.368734 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.372387 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.373199 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.373205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.373208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.373367 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.373959 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.373967 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.373972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.374500 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.374626 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.375686 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.375970 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":257,"threads_started":1}
I20260926 21:29:01.377458 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.354274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45222 (local address 127.14.142.60:36139)
0926 21:29:01.355073 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.355084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.355127 (+ 43us) server_negotiation.cc:407] Connection header received
0926 21:29:01.356854 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.356877 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.357307 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.357628 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.359458 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.361179 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.362910 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.363669 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.363817 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.368361 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.368369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.368382 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.368457 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.372585 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.373346 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.373353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.373355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.373578 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.374402 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.374414 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.374416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.375186 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.375519 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.375959 (+ 440us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.376172 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":577,"spinlock_wait_cycles":95104,"thread_start_us":360,"threads_started":1}
I20260926 21:29:01.377478 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.358907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:45909 (local address 127.0.0.1:43630)
0926 21:29:01.359873 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.359915 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:01.360100 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:01.360612 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:01.360620 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:01.361280 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:01.361533 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.361542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.362977 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.362981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.364313 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.364329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.365427 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.365433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.365558 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.366171 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:01.366197 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:01.368461 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:01.372611 (+ 4150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.372624 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.372643 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.373067 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.373498 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.373502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.373505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.373729 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.374644 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:01.374654 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:01.375186 (+ 532us) client_negotiation.cc:769] Sending connection context
0926 21:29:01.375505 (+ 319us) client_negotiation.cc:240] Negotiation successful
0926 21:29:01.375973 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":43,"spinlock_wait_cycles":1408,"thread_start_us":539,"threads_started":1}
I20260926 21:29:01.379195 5112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458141, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:29:01.393744 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.379920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:43363 (local address 127.0.0.1:44840)
0926 21:29:01.380120 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.380143 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:01.380279 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:01.381261 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:01.381265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:01.381597 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:01.381736 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.381744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.382756 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.382762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.383638 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.383648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.384336 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.384341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.384523 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.385164 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:01.385181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:01.387502 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:01.391167 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.391177 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.391184 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.391730 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.392182 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.392189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.392196 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.392388 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.393032 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:01.393035 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:01.393170 (+ 135us) client_negotiation.cc:769] Sending connection context
0926 21:29:01.393294 (+ 124us) client_negotiation.cc:240] Negotiation successful
0926 21:29:01.393507 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 21:29:01.394127 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.380060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.14.142.61:43363)
0926 21:29:01.380787 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.380794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.380813 (+ 19us) server_negotiation.cc:407] Connection header received
0926 21:29:01.380875 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.380883 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.380967 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.381075 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.381922 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.382620 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.383776 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.384194 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.384325 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.387720 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.387726 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.387730 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.387764 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.390940 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.391845 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.391850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.391852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.391991 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.392521 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.392527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.392531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.392913 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.393075 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.393436 (+ 361us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.393673 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":492,"threads_started":1}
I20260926 21:29:01.395354 5112 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:01.398041 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.381158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.14.142.62:45909)
0926 21:29:01.381331 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.381340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.381360 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:29:01.381579 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.381585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.381668 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.381779 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.383070 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.383833 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.385297 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.385738 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.385888 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.390285 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.390291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.390294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.390329 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.393862 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.394695 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.394699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.394701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.395128 (+ 427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.396155 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.396163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.396168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.396772 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.397166 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.397414 (+ 248us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.397679 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458141, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:29:01.411957 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.396264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.14.142.60:36139)
0926 21:29:01.396443 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.396447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.396788 (+ 341us) server_negotiation.cc:407] Connection header received
0926 21:29:01.397008 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.397016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.397130 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.397275 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.398171 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.399215 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.400997 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.401598 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.401703 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.405881 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.405888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.405893 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.405932 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.408820 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.409742 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.409749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.409759 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.409889 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.410354 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.410357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.410360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.410654 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.410827 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.411221 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.411506 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":57}
I20260926 21:29:01.412046 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.396510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45224)
0926 21:29:01.396704 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.396720 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:01.396845 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:01.397303 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:01.397310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:01.397833 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:01.398016 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.398025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.399357 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.399366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.400699 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.400713 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.401883 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.401890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.402220 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.402934 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:01.402965 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:01.405660 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:01.408962 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.408969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.408974 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.409590 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.410037 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.410043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.410047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.410193 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.410799 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:01.410804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:01.410960 (+ 156us) client_negotiation.cc:769] Sending connection context
0926 21:29:01.411121 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 21:29:01.411664 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":196}
I20260926 21:29:01.417908 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.400490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44854 (local address 127.14.142.61:43363)
0926 21:29:01.400739 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.400746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.400768 (+ 22us) server_negotiation.cc:407] Connection header received
0926 21:29:01.400997 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.401002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.401081 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.401181 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.403056 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.403801 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.405481 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.406017 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.406155 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.411323 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.411329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.411333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.411372 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.414676 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.415545 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.415549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.415552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.415698 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.416263 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.416270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.416273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.416841 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.417080 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.417418 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.417616 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 21:29:01.418442 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: Bootstrap starting.
I20260926 21:29:01.422147 5112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: Neither blocks nor log segments found. Creating new log.
I20260926 21:29:01.423319 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:29:01.423369 5112 log.cc:824] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: Log is configured to *not* fsync() on all Append() calls
I20260926 21:29:01.426369 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: No bootstrap required, opened a new log
I20260926 21:29:01.426316 4938 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "dd656a670d7546db985f4ae7ec2d8615"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43642
I20260926 21:29:01.433902 5112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.434275 5112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d5b9d277cd34add9f768603b74341a6, State: Initialized, Role: FOLLOWER
I20260926 21:29:01.434733 5112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.435724 5120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.436116 5120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.436923 5112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 21:29:01.448283 5131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:29:01.448457 5131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:29:01.445951 5045 master_runner.cc:186] Error getting master registration for 127.14.142.62:45909: OK, instance_id { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" instance_seqno: 1790458140100818 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 21:29:01.459507 5045 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 21:29:01.469077 5037 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 5
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:29:01.485713 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.469996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.14.142.60:36139)
0926 21:29:01.470184 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.470192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.470216 (+ 24us) server_negotiation.cc:407] Connection header received
0926 21:29:01.470469 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.470475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.470566 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.470676 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.471505 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.472350 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.473610 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.474118 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.474264 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.478613 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.478620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.478624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.478665 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.482443 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.483365 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.483373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.483378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.483526 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.483946 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.483952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.483958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.484456 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.484654 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.485079 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.485318 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 21:29:01.485739 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.469903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45226)
0926 21:29:01.470130 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.470151 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:01.470292 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:01.470717 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:01.470723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:01.471173 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:01.471335 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.471343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.472507 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.472515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.473459 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.473470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.474274 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.474281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.474558 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.475262 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:01.475282 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:01.478360 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:01.482622 (+ 4262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.482629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.482635 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.483220 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.483649 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.483654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.483658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.483832 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.484625 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:01.484630 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:01.484799 (+ 169us) client_negotiation.cc:769] Sending connection context
0926 21:29:01.484966 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 21:29:01.485303 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260926 21:29:01.491814 5037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: Bootstrap starting.
I20260926 21:29:01.495316 5037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: Neither blocks nor log segments found. Creating new log.
I20260926 21:29:01.496382 5037 log.cc:824] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: Log is configured to *not* fsync() on all Append() calls
I20260926 21:29:01.499152 5037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: No bootstrap required, opened a new log
I20260926 21:29:01.506600 5037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.506965 5037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23599be46ee745bab7e77db8faaab722, State: Initialized, Role: FOLLOWER
I20260926 21:29:01.507450 5037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.508453 5134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.508855 5134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.509681 5037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 21:29:01.522467 5145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:29:01.522743 5145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:29:01.519140 4970 master_runner.cc:186] Error getting master registration for 127.14.142.62:45909: OK, instance_id { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" instance_seqno: 1790458140100818 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 21:29:01.532522 4970 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 21:29:01.539148 4963 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } attempt: 5
Sep 26 21:29:01 dist-test-slave-4d5t krb5kdc[4882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458140, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:29:01.559006 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.539947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:36139 (local address 127.0.0.1:45238)
0926 21:29:01.540390 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:01.540412 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:01.540583 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:01.541573 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:01.541579 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:01.542120 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:01.542344 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.542357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.543909 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.543914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.545519 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:01.545536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.546797 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.546804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:01.547024 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.547779 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:01.547803 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:01.551060 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:01.555894 (+ 4834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.555900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.555905 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.556502 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.557003 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:01.557009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:01.557013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:01.557208 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:01.557866 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:01.557871 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:01.558063 (+ 192us) client_negotiation.cc:769] Sending connection context
0926 21:29:01.558222 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 21:29:01.558545 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298}
I20260926 21:29:01.559006 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.540813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.14.142.60:36139)
0926 21:29:01.541004 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.541013 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.541033 (+ 20us) server_negotiation.cc:407] Connection header received
0926 21:29:01.541100 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.541105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.541207 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.541310 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.542552 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.543724 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.545722 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.546437 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.546572 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.551347 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.551364 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.551371 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.551418 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.555696 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.556682 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.556688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.556692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.556863 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.557346 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.557352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.557355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.557714 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.557868 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.558376 (+ 508us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.558629 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:29:01.565472 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615: Bootstrap starting.
I20260926 21:29:01.569871 4963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615: Neither blocks nor log segments found. Creating new log.
W20260926 21:29:01.571412 5066 tablet.cc:2369] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:29:01.571497 4963 log.cc:824] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615: Log is configured to *not* fsync() on all Append() calls
I20260926 21:29:01.587854 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615: No bootstrap required, opened a new log
I20260926 21:29:01.596380 4963 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.596894 4963 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd656a670d7546db985f4ae7ec2d8615, State: Initialized, Role: FOLLOWER
I20260926 21:29:01.597524 4963 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.599140 5148 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.600066 4963 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:29:01.600136 4938 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 21:29:01.600337 4938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:29:01.604211 4938 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.607121 5012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd656a670d7546db985f4ae7ec2d8615" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "23599be46ee745bab7e77db8faaab722"
I20260926 21:29:01.607700 5012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:29:01.612013 5012 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd656a670d7546db985f4ae7ec2d8615 in term 1.
I20260926 21:29:01.613390 4911 leader_election.cc:302] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23599be46ee745bab7e77db8faaab722, dd656a670d7546db985f4ae7ec2d8615; no voters:
I20260926 21:29:01.616926 5087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd656a670d7546db985f4ae7ec2d8615" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3d5b9d277cd34add9f768603b74341a6"
I20260926 21:29:01.617462 5087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:29:01.621212 5087 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd656a670d7546db985f4ae7ec2d8615 in term 1.
I20260926 21:29:01.622323 4938 leader_election.cc:288] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [CANDIDATE]: Term 1 election: Requested vote from peers 23599be46ee745bab7e77db8faaab722 (127.14.142.61:43363), 3d5b9d277cd34add9f768603b74341a6 (127.14.142.60:36139)
I20260926 21:29:01.599965 5148 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.637239 5148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:29:01.638123 5148 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [term 1 LEADER]: Becoming Leader. State: Replica: dd656a670d7546db985f4ae7ec2d8615, State: Running, Role: LEADER
I20260926 21:29:01.639300 5148 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } }
I20260926 21:29:01.651603 5151 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd656a670d7546db985f4ae7ec2d8615. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.652194 5151 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: This master's current role is: LEADER
I20260926 21:29:01.653165 5156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:29:01.701038 5156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260926 21:29:01.714660 4991 tablet.cc:2369] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:29:01.744879 5171 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 21:29:01.753569 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.661622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.14.142.62:45909)
0926 21:29:01.661795 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.661801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.669941 (+ 8140us) server_negotiation.cc:407] Connection header received
0926 21:29:01.670215 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.670223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.670330 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.670404 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.690613 (+ 20209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.692297 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.703714 (+ 11417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.704587 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.704725 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.723183 (+ 18458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.723193 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.723199 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.723278 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.727518 (+ 4240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.740020 (+ 12502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.740028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.740032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.740231 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.745898 (+ 5667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.745927 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.745931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.746416 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.746734 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.751786 (+ 5052us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.752166 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 21:29:01.758908 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.664877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.14.142.60:36139)
0926 21:29:01.665114 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.665121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.671446 (+ 6325us) server_negotiation.cc:407] Connection header received
0926 21:29:01.671680 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.671687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.671784 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.671859 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.681001 (+ 9142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.682165 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.701442 (+ 19277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.702240 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.702377 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.729163 (+ 26786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.729172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.729178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.729235 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.733287 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.744770 (+ 11483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.744779 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.744783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.745139 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.750162 (+ 5023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.750169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.750173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.750776 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.751080 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.758206 (+ 7126us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.758513 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 21:29:01.762303 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:01.660779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.14.142.61:43363)
0926 21:29:01.660961 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:01.660969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:01.666455 (+ 5486us) server_negotiation.cc:407] Connection header received
0926 21:29:01.666759 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:01.666783 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:01.666874 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:01.666952 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:01.685068 (+ 18116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.686380 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.697336 (+ 10956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:01.698108 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:01.698227 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:01.734406 (+ 36179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:01.734415 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:01.734421 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:01.734485 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:01.738877 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.747320 (+ 8443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.747333 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.747338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.747527 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:01.754366 (+ 6839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:01.754374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:01.754379 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:01.754994 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:01.755447 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:01.761610 (+ 6163us) server_negotiation.cc:299] Negotiation successful
0926 21:29:01.761877 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260926 21:29:01.773793 5087 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [term 1 FOLLOWER]: Refusing update from remote peer dd656a670d7546db985f4ae7ec2d8615: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:29:01.775974 5012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [term 1 FOLLOWER]: Refusing update from remote peer dd656a670d7546db985f4ae7ec2d8615: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:29:01.777578 5148 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:29:01.782505 5148 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:29:01.823746 5134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd656a670d7546db985f4ae7ec2d8615. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.824128 5134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.828239 5120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd656a670d7546db985f4ae7ec2d8615. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.828593 5120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.839989 5173 mvcc.cc:205] Tried to move back new op lower bound from 7333716548691398656 to 7333716548182360064. Current Snapshot: MvccSnapshot[applied={T|T < 7333716548691398656}]
I20260926 21:29:01.841104 5134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.841380 5134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.856316 5172 mvcc.cc:205] Tried to move back new op lower bound from 7333716548691398656 to 7333716548182360064. Current Snapshot: MvccSnapshot[applied={T|T < 7333716548691398656}]
I20260926 21:29:01.866631 5120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.866973 5120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:01.871613 5156 catalog_manager.cc:1347] Generated new cluster ID: b1f87e873153460ca6fb10f0e14a7cdf
I20260926 21:29:01.871770 5156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:29:01.873787 5151 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.874097 5151 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: This master's current role is: LEADER
I20260926 21:29:01.874377 5151 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd656a670d7546db985f4ae7ec2d8615" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd656a670d7546db985f4ae7ec2d8615" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 45909 } } peers { permanent_uuid: "23599be46ee745bab7e77db8faaab722" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 43363 } } peers { permanent_uuid: "3d5b9d277cd34add9f768603b74341a6" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 36139 } } }
I20260926 21:29:01.874603 5151 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615 [sys.catalog]: This master's current role is: LEADER
I20260926 21:29:01.917779 5156 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:29:01.919979 5156 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:29:01.964887 5156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd656a670d7546db985f4ae7ec2d8615: Generated new TSK 0
I20260926 21:29:01.966866 5156 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 21:29:02.125442 5116 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.108069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43654 (local address 127.14.142.62:45909)
0926 21:29:02.108271 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.108277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.123004 (+ 14727us) server_negotiation.cc:407] Connection header received
0926 21:29:02.123705 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.123712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.123809 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.123904 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.125174 (+ 1270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43654: BlockingRecv error: recv got EOF from 127.0.0.1:43654 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260926 21:29:02.127435 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.108318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.14.142.61:43363)
0926 21:29:02.108573 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.108580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.122439 (+ 13859us) server_negotiation.cc:407] Connection header received
0926 21:29:02.125896 (+ 3457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.125903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.125992 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.126067 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.127196 (+ 1129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44862: BlockingRecv error: recv got EOF from 127.0.0.1:44862 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 21:29:02.130302 5115 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.116418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.14.142.60:36139)
0926 21:29:02.116645 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.116652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.128457 (+ 11805us) server_negotiation.cc:407] Connection header received
0926 21:29:02.128838 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.128845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.128964 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.129041 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.130057 (+ 1016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45268: BlockingRecv error: recv got EOF from 127.0.0.1:45268 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 21:29:02.159089 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.154885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.14.142.61:43363)
0926 21:29:02.155094 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.155101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.157177 (+ 2076us) server_negotiation.cc:407] Connection header received
0926 21:29:02.157418 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.157426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.157568 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.157696 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.158746 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44874: BlockingRecv error: recv got EOF from 127.0.0.1:44874 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260926 21:29:02.171092 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.166360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.14.142.61:43363)
0926 21:29:02.166550 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.166556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.169302 (+ 2746us) server_negotiation.cc:407] Connection header received
0926 21:29:02.169519 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.169526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.169627 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.169764 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.170757 (+ 993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44882: BlockingRecv error: recv got EOF from 127.0.0.1:44882 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260926 21:29:02.186540 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.183267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.14.142.61:43363)
0926 21:29:02.183490 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.183497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.185134 (+ 1637us) server_negotiation.cc:407] Connection header received
0926 21:29:02.185304 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.185311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.185410 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.185546 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.186260 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44892: BlockingRecv error: recv got EOF from 127.0.0.1:44892 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 21:29:02.195899 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.193835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44900 (local address 127.14.142.61:43363)
0926 21:29:02.194098 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.194107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.194444 (+ 337us) server_negotiation.cc:407] Connection header received
0926 21:29:02.194673 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.194679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.194763 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.194876 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.195650 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44900: BlockingRecv error: recv got EOF from 127.0.0.1:44900 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260926 21:29:02.209564 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.207604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44912 (local address 127.14.142.61:43363)
0926 21:29:02.207806 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.207812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.208117 (+ 305us) server_negotiation.cc:407] Connection header received
0926 21:29:02.208284 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.208291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.208409 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.208542 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.209301 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44912: BlockingRecv error: recv got EOF from 127.0.0.1:44912 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 21:29:02.231115 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.229244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.14.142.61:43363)
0926 21:29:02.229478 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.229486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.229655 (+ 169us) server_negotiation.cc:407] Connection header received
0926 21:29:02.229900 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.229904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.229977 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.230094 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.230843 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44926: BlockingRecv error: recv got EOF from 127.0.0.1:44926 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260926 21:29:02.268790 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.266811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.14.142.61:43363)
0926 21:29:02.267032 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.267041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.267282 (+ 241us) server_negotiation.cc:407] Connection header received
0926 21:29:02.267550 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.267555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.267628 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.267741 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.268508 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44928: BlockingRecv error: recv got EOF from 127.0.0.1:44928 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260926 21:29:02.338483 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.336536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.14.142.61:43363)
0926 21:29:02.336782 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.336787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.337207 (+ 420us) server_negotiation.cc:407] Connection header received
0926 21:29:02.337381 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.337385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.337465 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.337568 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.338230 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44930: BlockingRecv error: recv got EOF from 127.0.0.1:44930 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
I20260926 21:29:02.450803 5131 catalog_manager.cc:1259] Loaded cluster ID: b1f87e873153460ca6fb10f0e14a7cdf
I20260926 21:29:02.450945 5131 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: loading cluster ID for follower catalog manager: success
I20260926 21:29:02.453945 5131 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: acquiring CA information for follower catalog manager: success
I20260926 21:29:02.455976 5131 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d5b9d277cd34add9f768603b74341a6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 21:29:02.472430 5118 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:02.470254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.14.142.61:43363)
0926 21:29:02.470523 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.470530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.470853 (+ 323us) server_negotiation.cc:407] Connection header received
0926 21:29:02.471062 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.471068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.471161 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.471318 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:02.472137 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44944: BlockingRecv error: recv got EOF from 127.0.0.1:44944 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 21:29:02.524904 5145 catalog_manager.cc:1259] Loaded cluster ID: b1f87e873153460ca6fb10f0e14a7cdf
I20260926 21:29:02.525017 5145 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: loading cluster ID for follower catalog manager: success
I20260926 21:29:02.527544 5145 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: acquiring CA information for follower catalog manager: success
I20260926 21:29:02.529482 5145 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23599be46ee745bab7e77db8faaab722: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:29:02.745065 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:02.738355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44948 (local address 127.14.142.61:43363)
0926 21:29:02.738586 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.738594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.739035 (+ 441us) server_negotiation.cc:407] Connection header received
0926 21:29:02.739312 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.739317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.739497 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.739637 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:02.740152 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.741025 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.742537 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.743015 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.743172 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:02.743734 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:02.744365 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:02.744539 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:02.744655 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 21:29:02.744748 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 21:29:02.758781 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:02.752015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.14.142.60:36139)
0926 21:29:02.753103 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.753108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.753126 (+ 18us) server_negotiation.cc:407] Connection header received
0926 21:29:02.753226 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.753230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.753407 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.753573 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:02.754047 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.754893 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.756382 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.756855 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.756963 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:02.757481 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:02.758004 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:02.758118 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:02.758290 (+ 172us) server_negotiation.cc:299] Negotiation successful
0926 21:29:02.758416 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":849,"threads_started":1}
I20260926 21:29:02.775804 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:02.766782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.14.142.61:43363)
0926 21:29:02.766985 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.766992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.767936 (+ 944us) server_negotiation.cc:407] Connection header received
0926 21:29:02.768218 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.768227 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.768373 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.768477 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:02.769457 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.770762 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.773207 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.774043 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.774185 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:02.774732 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:02.774911 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:02.775031 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:02.775356 (+ 325us) server_negotiation.cc:299] Negotiation successful
0926 21:29:02.775457 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 21:29:02.776867 5243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:02.766647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.14.142.60:36139)
0926 21:29:02.766985 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.766992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.769123 (+ 2131us) server_negotiation.cc:407] Connection header received
0926 21:29:02.769382 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.769389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.769492 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.769607 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:02.770560 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.771737 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.774129 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.774856 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.775024 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:02.775928 (+ 904us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:02.776083 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:02.776180 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:02.776432 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 21:29:02.776539 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260926 21:29:02.777055 5248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:02.766163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.14.142.62:45909)
0926 21:29:02.766792 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:02.766803 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:02.768301 (+ 1498us) server_negotiation.cc:407] Connection header received
0926 21:29:02.768701 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:02.768707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:02.769004 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:02.769292 (+ 288us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:02.770226 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.771534 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.773951 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:02.774643 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:02.774837 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:02.775461 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:02.776192 (+ 731us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:02.776312 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:02.776573 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 21:29:02.776678 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":679,"threads_started":1}
I20260926 21:29:02.783774 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4895
I20260926 21:29:02.785120 4958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:29:02.895340 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4895
I20260926 21:29:02.918335 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 4970
I20260926 21:29:02.919584 5032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:29:03.033273 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 4970
I20260926 21:29:03.058378 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 5045
I20260926 21:29:03.059808 5107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:29:03.169370 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 5045
2026-09-26T21:29:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5057 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 21:29:03 dist-test-slave-4d5t krb5kdc[5257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 21:29:03 dist-test-slave-4d5t krb5kdc[5257](info): set up 2 sockets
Sep 26 21:29:03 dist-test-slave-4d5t krb5kdc[5257](info): commencing operation
krb5kdc: starting...
W20260926 21:29:04.318136 14904 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.008s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/test-user.keytab.
Sep 26 21:29:04 dist-test-slave-4d5t krb5kdc[5257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T21:29:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T21:29:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.14.142.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
Entry for principal kudu/127.14.142.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab.
I20260926 21:29:04.620201 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:34719
--webserver_interface=127.14.142.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:34719,127.14.142.61:36751,127.14.142.60:41985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 21:29:04.980535 5270 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 21:29:04.980986 5270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:29:04.981132 5270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:29:04.981268 5270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:29:04.989048 5270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:29:04.989164 5270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:29:04.989228 5270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:29:04.989318 5270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:29:04.989566 5270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:29:05.002722 5270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:34719,127.14.142.61:36751,127.14.142.60:41985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.62.keytab
--principal=kudu/127.14.142.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.62:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.14.142.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:29:05.004786 5270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:29:05.006500 5270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:29:05.020525 5275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:05.020545 5276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:05.020454 5278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:29:05.021592 5270 server_base.cc:1034] running on GCE node
I20260926 21:29:05.022444 5270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:29:05.024160 5270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:29:05.025486 5270 hybrid_clock.cc:648] HybridClock initialized: now 1790458145025393 us; error 39 us; skew 500 ppm
I20260926 21:29:05.025923 5270 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:29:05 dist-test-slave-4d5t krb5kdc[5257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:29:05.029778 5270 init.cc:377] Logged in from keytab as kudu/127.14.142.62@KRBTEST.COM (short username kudu)
I20260926 21:29:05.031670 5270 webserver.cc:460] Webserver started at http://127.14.142.62:41427/ using document root <none> and password file <none>
I20260926 21:29:05.032431 5270 fs_manager.cc:359] Metadata directory not provided
I20260926 21:29:05.032529 5270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:29:05.032899 5270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:29:05.036250 5270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data/instance:
uuid: "280ed4fc559a4f0ab1484b2d7516c2d1"
format_stamp: "Formatted at 2026-09-26 21:29:05 on dist-test-slave-4d5t"
server_key: "527c9e35ca4a6e71ad1613c0ee26682c"
server_key_iv: "3cc62d23b4b177a2de1cfafd38f52e2d"
server_key_version: "encryptionkey@0"
I20260926 21:29:05.037214 5270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal/instance:
uuid: "280ed4fc559a4f0ab1484b2d7516c2d1"
format_stamp: "Formatted at 2026-09-26 21:29:05 on dist-test-slave-4d5t"
server_key: "527c9e35ca4a6e71ad1613c0ee26682c"
server_key_iv: "3cc62d23b4b177a2de1cfafd38f52e2d"
server_key_version: "encryptionkey@0"
I20260926 21:29:05.043776 5270 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:29:05.048431 5285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:29:05.050503 5270 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 21:29:05.050755 5270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal
uuid: "280ed4fc559a4f0ab1484b2d7516c2d1"
format_stamp: "Formatted at 2026-09-26 21:29:05 on dist-test-slave-4d5t"
server_key: "527c9e35ca4a6e71ad1613c0ee26682c"
server_key_iv: "3cc62d23b4b177a2de1cfafd38f52e2d"
server_key_version: "encryptionkey@0"
I20260926 21:29:05.051036 5270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:29:05.082399 5270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:29:05.087142 5270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:29:05.087467 5270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:29:05.108318 5270 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.62:34719
I20260926 21:29:05.108338 5336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.62:34719 every 8 connection(s)
I20260926 21:29:05.109822 5270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/data/info.pb
I20260926 21:29:05.112700 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 5270
I20260926 21:29:05.112910 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-0/wal/instance
I20260926 21:29:05.113366 14904 external_mini_cluster.cc:1428] Setting key 7856b41fe060445b873c39eac40c4206
I20260926 21:29:05.114955 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:29:05.121956 5337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:29:05 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.14.142.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.61@KRBTEST.COM" created.
I20260926 21:29:05.151350 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.125842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34719 (local address 127.0.0.1:40796)
0926 21:29:05.126793 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.126849 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:05.129854 (+ 3005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:05.130931 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:05.130945 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:05.131840 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:05.132372 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.132401 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.134138 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.134146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.136188 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.136210 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.137417 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.137429 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.137653 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.138385 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:05.138423 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:05.141595 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:05.146064 (+ 4469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.146080 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.146104 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.146598 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.147121 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.147133 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.147135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.147321 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.148282 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:05.148291 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:05.148536 (+ 245us) client_negotiation.cc:769] Sending connection context
0926 21:29:05.148950 (+ 414us) client_negotiation.cc:240] Negotiation successful
0926 21:29:05.149651 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":170,"thread_start_us":353,"threads_started":1}
I20260926 21:29:05.151374 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.127043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.14.142.62:34719)
0926 21:29:05.127682 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:05.127687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:05.127732 (+ 45us) server_negotiation.cc:407] Connection header received
0926 21:29:05.130337 (+ 2605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:05.130346 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:05.130662 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:05.130858 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:05.132705 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.133952 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.136391 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.137102 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.137267 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.141922 (+ 4655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:05.141931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:05.141946 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:05.141982 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:05.145873 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.146776 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.146789 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.146798 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.146973 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.147465 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.147473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.147483 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.148086 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:05.148335 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:05.149094 (+ 759us) server_negotiation.cc:299] Negotiation successful
0926 21:29:05.149506 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":464,"thread_start_us":326,"threads_started":1}
I20260926 21:29:05.155224 5337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:05.156862 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.156211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:34642)
0926 21:29:05.156447 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.156662 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 21:29:05.157320 5286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
I20260926 21:29:05.159196 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.158609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:34644)
0926 21:29:05.158841 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.158961 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 21:29:05.159485 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:36751: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab.
Entry for principal kudu/127.14.142.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab.
I20260926 21:29:05.180531 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:36751
--webserver_interface=127.14.142.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab
--principal=kudu/127.14.142.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:34719,127.14.142.61:36751,127.14.142.60:41985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:29:05.208904 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } attempt: 1
I20260926 21:29:05.210327 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.209775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:34654)
0926 21:29:05.209999 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.210122 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260926 21:29:05.212028 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.211490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:34670)
0926 21:29:05.211768 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.211870 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260926 21:29:05.212385 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:36751: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
I20260926 21:29:05.280786 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } attempt: 2
I20260926 21:29:05.282518 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.281831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:34676)
0926 21:29:05.282231 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.282348 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260926 21:29:05.283667 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.283270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:34682)
0926 21:29:05.283445 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.283518 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 21:29:05.283881 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:36751: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
I20260926 21:29:05.375167 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } attempt: 3
I20260926 21:29:05.376448 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.376000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54132)
0926 21:29:05.376201 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.376308 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 21:29:05.377490 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.377126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54136)
0926 21:29:05.377286 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.377355 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 21:29:05.377679 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:36751: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
I20260926 21:29:05.520948 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } attempt: 4
I20260926 21:29:05.522251 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.521657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54148)
0926 21:29:05.522016 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.522110 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260926 21:29:05.523334 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.522940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54150)
0926 21:29:05.523100 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.523195 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 21:29:05.523572 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.61:36751: Network error: Client connection negotiation failed: client connection to 127.14.142.61:36751: connect: Connection refused (error 111)
W20260926 21:29:05.554152 5344 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 21:29:05.554550 5344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:29:05.554677 5344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:29:05.554818 5344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:29:05.562506 5344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:29:05.562644 5344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:29:05.562767 5344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:29:05.562866 5344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:29:05.562978 5344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:29:05.573350 5344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:34719,127.14.142.61:36751,127.14.142.60:41985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.61.keytab
--principal=kudu/127.14.142.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.61:36751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.14.142.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:29:05.574766 5344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:29:05.576157 5344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:29:05.589032 5350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:05.589087 5352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:05.588994 5349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:29:05.589877 5344 server_base.cc:1034] running on GCE node
I20260926 21:29:05.590739 5344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:29:05.592377 5344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:29:05.593658 5344 hybrid_clock.cc:648] HybridClock initialized: now 1790458145593605 us; error 61 us; skew 500 ppm
I20260926 21:29:05.594030 5344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:29:05 dist-test-slave-4d5t krb5kdc[5257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:29:05.597606 5344 init.cc:377] Logged in from keytab as kudu/127.14.142.61@KRBTEST.COM (short username kudu)
I20260926 21:29:05.599450 5344 webserver.cc:460] Webserver started at http://127.14.142.61:39687/ using document root <none> and password file <none>
I20260926 21:29:05.600241 5344 fs_manager.cc:359] Metadata directory not provided
I20260926 21:29:05.600389 5344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:29:05.600829 5344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:29:05.604226 5344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data/instance:
uuid: "cebf83bacad94cbfb0e25f208d286a50"
format_stamp: "Formatted at 2026-09-26 21:29:05 on dist-test-slave-4d5t"
server_key: "2f65d5ab7bb2b804ae019011184fd053"
server_key_iv: "5784f3d2d3b6e6c82512c9a8c9f41bf9"
server_key_version: "encryptionkey@0"
I20260926 21:29:05.605227 5344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal/instance:
uuid: "cebf83bacad94cbfb0e25f208d286a50"
format_stamp: "Formatted at 2026-09-26 21:29:05 on dist-test-slave-4d5t"
server_key: "2f65d5ab7bb2b804ae019011184fd053"
server_key_iv: "5784f3d2d3b6e6c82512c9a8c9f41bf9"
server_key_version: "encryptionkey@0"
I20260926 21:29:05.611809 5344 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 21:29:05.616262 5359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:29:05.617905 5344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 21:29:05.618153 5344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal
uuid: "cebf83bacad94cbfb0e25f208d286a50"
format_stamp: "Formatted at 2026-09-26 21:29:05 on dist-test-slave-4d5t"
server_key: "2f65d5ab7bb2b804ae019011184fd053"
server_key_iv: "5784f3d2d3b6e6c82512c9a8c9f41bf9"
server_key_version: "encryptionkey@0"
I20260926 21:29:05.618428 5344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:29:05.634800 5344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:29:05.639178 5344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:29:05.639482 5344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:29:05.660620 5344 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.61:36751
I20260926 21:29:05.660537 5410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.61:36751 every 8 connection(s)
I20260926 21:29:05.662189 5344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/data/info.pb
I20260926 21:29:05.666740 5411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:29:05.671109 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 5344
I20260926 21:29:05.671350 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-1/wal/instance
I20260926 21:29:05.671849 14904 external_mini_cluster.cc:1428] Setting key 054fff815198922e842bba3b3265fa79
I20260926 21:29:05.673645 5411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 21:29:05 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:29:05.695312 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.676773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.14.142.62:34719)
0926 21:29:05.677496 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:05.677502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:05.678422 (+ 920us) server_negotiation.cc:407] Connection header received
0926 21:29:05.680444 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:05.680452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:05.680605 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:05.680732 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:05.682437 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.683201 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.684821 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.685252 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.685398 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.689044 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:05.689049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:05.689053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:05.689088 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:05.691910 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.692861 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.692865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.692868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.692994 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.693451 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.693460 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.693462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.693753 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:05.693918 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:05.694683 (+ 765us) server_negotiation.cc:299] Negotiation successful
0926 21:29:05.694920 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":440,"threads_started":1}
I20260926 21:29:05.696355 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.677249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34719 (local address 127.0.0.1:34228)
0926 21:29:05.678208 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.678249 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:05.680117 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:05.681011 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:05.681020 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:05.681604 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:05.682193 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.682211 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.683368 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.683374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.684675 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.684688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.685493 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.685500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.685752 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.686490 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:05.686513 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:05.688789 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:05.692095 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.692105 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.692127 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.692716 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.693120 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.693125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.693129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.693320 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.693896 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:05.693905 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:05.694307 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 21:29:05.694583 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 21:29:05.695146 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":439,"threads_started":1}
WARNING: no policy specified for kudu/127.14.142.60@KRBTEST.COM; defaulting to no policy
I20260926 21:29:05.698432 5411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.14.142.60@KRBTEST.COM" created.
Sep 26 21:29:05 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:29:05.716796 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.699348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54158)
0926 21:29:05.699653 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.699684 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:05.699844 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:05.701224 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:05.701230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:05.701734 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:05.701875 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.701884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.703417 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.703425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.704379 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.704391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.705343 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.705349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.705585 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.706549 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:05.706577 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:05.709282 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:05.714120 (+ 4838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.714128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.714134 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.714555 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.714969 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.714975 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.714978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.715123 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.715737 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:05.715747 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:05.715884 (+ 137us) client_negotiation.cc:769] Sending connection context
0926 21:29:05.716031 (+ 147us) client_negotiation.cc:240] Negotiation successful
0926 21:29:05.716363 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260926 21:29:05.716943 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.699605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.14.142.61:36751)
0926 21:29:05.700518 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:05.700525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:05.700673 (+ 148us) server_negotiation.cc:407] Connection header received
0926 21:29:05.700772 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:05.700779 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:05.701039 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:05.701241 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:05.702066 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.703268 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.704532 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.705182 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.705334 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.709581 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:05.709604 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:05.709645 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:05.709682 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:05.713928 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.714704 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.714719 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.714721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.714848 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.715227 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.715235 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.715238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.715604 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:05.715826 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:05.716203 (+ 377us) server_negotiation.cc:299] Negotiation successful
0926 21:29:05.716492 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":718,"thread_start_us":513,"threads_started":1}
I20260926 21:29:05.719897 5411 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:05.721293 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.720691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33302)
0926 21:29:05.720892 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.721060 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 21:29:05.721604 5360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:05.723582 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.723115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33312)
0926 21:29:05.723301 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.723419 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 21:29:05.723915 5411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.14.142.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab.
Entry for principal kudu/127.14.142.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab.
I20260926 21:29:05.744513 14904 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
/tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:41985
--webserver_interface=127.14.142.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab
--principal=kudu/127.14.142.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.14.142.20:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.14.142.62:34719,127.14.142.61:36751,127.14.142.60:41985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 21:29:05.773403 5411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 1
I20260926 21:29:05.774750 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.774109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33320)
0926 21:29:05.774377 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.774530 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 21:29:05.775938 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.775452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33334)
0926 21:29:05.775650 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.775759 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 21:29:05.776217 5411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:05.822896 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } attempt: 5
Sep 26 21:29:05 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:29:05.840284 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.823781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54174)
0926 21:29:05.824010 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.824036 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:05.824170 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:05.824813 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:05.824819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:05.825344 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:05.825606 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.825617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.827210 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.827217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.828201 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:05.828218 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.829255 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.829260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:05.829579 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.830309 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:05.830337 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:05.833250 (+ 2913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:05.837387 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.837392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.837396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.837882 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.838320 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:05.838324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:05.838327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:05.838476 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:05.839235 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:05.839239 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:05.839393 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 21:29:05.839532 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 21:29:05.839899 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 21:29:05.840490 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.823903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54174 (local address 127.14.142.61:36751)
0926 21:29:05.824130 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:05.824136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:05.824163 (+ 27us) server_negotiation.cc:407] Connection header received
0926 21:29:05.824350 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:05.824356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:05.824441 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:05.824622 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:05.825843 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.826952 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.828396 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:05.829061 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:05.829198 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:05.833503 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:05.833511 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:05.833515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:05.833561 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:05.837175 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.838023 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.838029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.838032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.838187 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:05.838603 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:05.838611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:05.838616 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:05.839093 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:05.839275 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:05.839735 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 21:29:05.840057 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260926 21:29:05.841904 5337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:05.843075 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.842595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33338)
0926 21:29:05.842796 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.842897 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 21:29:05.844220 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.843805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33352)
0926 21:29:05.843967 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.844094 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 21:29:05.844446 5411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 2
W20260926 21:29:05.844491 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:05.845399 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.844961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33366)
0926 21:29:05.845141 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.845234 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 21:29:05.846407 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.846028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33382)
0926 21:29:05.846187 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.846270 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 21:29:05.846696 5411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:05.895823 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 1
I20260926 21:29:05.897248 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.896705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33398)
0926 21:29:05.896949 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.897052 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260926 21:29:05.898366 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.898010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33404)
0926 21:29:05.898165 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.898254 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 21:29:05.898583 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:05.937966 5411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 3
I20260926 21:29:05.939231 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.938726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33414)
0926 21:29:05.938964 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.939069 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 21:29:05.940447 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.939981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33428)
0926 21:29:05.940222 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.940316 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260926 21:29:05.940837 5411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:05.966842 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 2
I20260926 21:29:05.968256 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.967697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33438)
0926 21:29:05.967973 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.968077 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260926 21:29:05.969309 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:05.968956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33452)
0926 21:29:05.969117 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:05.969197 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 21:29:05.969502 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:06.075780 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 3
I20260926 21:29:06.077028 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.076539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33460)
0926 21:29:06.076767 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.076858 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260926 21:29:06.078061 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.077705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33472)
0926 21:29:06.077888 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.077955 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260926 21:29:06.078262 5337 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
I20260926 21:29:06.084169 5411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 4
I20260926 21:29:06.085366 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.084874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33484)
0926 21:29:06.085130 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.085210 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260926 21:29:06.086404 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.086086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33492)
0926 21:29:06.086234 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.086302 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 21:29:06.086691 5411 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.14.142.60:41985: Network error: Client connection negotiation failed: client connection to 127.14.142.60:41985: connect: Connection refused (error 111)
W20260926 21:29:06.120065 5419 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 21:29:06.120476 5419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 21:29:06.120631 5419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 21:29:06.120784 5419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 21:29:06.128363 5419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 21:29:06.128506 5419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 21:29:06.128638 5419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 21:29:06.128728 5419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 21:29:06.128821 5419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 21:29:06.139201 5419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.14.142.20:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.14.142.62:34719,127.14.142.61:36751,127.14.142.60:41985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/krb5kdc/kudu_127.14.142.60.keytab
--principal=kudu/127.14.142.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.14.142.60:41985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.14.142.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 20:26:36 UTC on ccycloud.quasar-bgclim.root.comops.site
build id 4675384
ASAN enabled
I20260926 21:29:06.140643 5419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 21:29:06.141974 5419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 21:29:06.154989 5424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:06.154913 5425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 21:29:06.155018 5427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 21:29:06.156239 5419 server_base.cc:1034] running on GCE node
I20260926 21:29:06.157233 5419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 21:29:06.158969 5419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 21:29:06.160216 5419 hybrid_clock.cc:648] HybridClock initialized: now 1790458146160147 us; error 54 us; skew 500 ppm
I20260926 21:29:06.160676 5419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 21:29:06.164252 5419 init.cc:377] Logged in from keytab as kudu/127.14.142.60@KRBTEST.COM (short username kudu)
I20260926 21:29:06.166088 5419 webserver.cc:460] Webserver started at http://127.14.142.60:45065/ using document root <none> and password file <none>
I20260926 21:29:06.166884 5419 fs_manager.cc:359] Metadata directory not provided
I20260926 21:29:06.166983 5419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 21:29:06.167256 5419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 21:29:06.171206 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data/instance:
uuid: "572317fe018d40d7b686fa5622042ef2"
format_stamp: "Formatted at 2026-09-26 21:29:06 on dist-test-slave-4d5t"
server_key: "e8285c5299852a435e85d37055eb09f7"
server_key_iv: "e18cf9c885ac29bfe49e7684a2588e8d"
server_key_version: "encryptionkey@0"
I20260926 21:29:06.172076 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal/instance:
uuid: "572317fe018d40d7b686fa5622042ef2"
format_stamp: "Formatted at 2026-09-26 21:29:06 on dist-test-slave-4d5t"
server_key: "e8285c5299852a435e85d37055eb09f7"
server_key_iv: "e18cf9c885ac29bfe49e7684a2588e8d"
server_key_version: "encryptionkey@0"
I20260926 21:29:06.178220 5419 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.005s
I20260926 21:29:06.182392 5434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 21:29:06.184723 5419 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260926 21:29:06.185024 5419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data,/tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal
uuid: "572317fe018d40d7b686fa5622042ef2"
format_stamp: "Formatted at 2026-09-26 21:29:06 on dist-test-slave-4d5t"
server_key: "e8285c5299852a435e85d37055eb09f7"
server_key_iv: "e18cf9c885ac29bfe49e7684a2588e8d"
server_key_version: "encryptionkey@0"
I20260926 21:29:06.185365 5419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 21:29:06.212757 5419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 21:29:06.217921 5419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 21:29:06.218221 5419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 21:29:06.237879 5419 rpc_server.cc:307] RPC server started. Bound to: 127.14.142.60:41985
I20260926 21:29:06.237919 5485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.14.142.60:41985 every 8 connection(s)
I20260926 21:29:06.239284 5419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/data/info.pb
I20260926 21:29:06.243523 5486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 21:29:06.248672 14904 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu as pid 5419
I20260926 21:29:06.248867 14904 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh_JUDQ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790457761955429-14904-0/minicluster-data/master-2/wal/instance
I20260926 21:29:06.249444 14904 external_mini_cluster.cc:1428] Setting key c2027678b3af006974aff95a7fc123dd
I20260926 21:29:06.251322 5486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:06.255463 5337 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 4
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.62@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:29:06.274396 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.256019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33512)
0926 21:29:06.256343 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.256364 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:06.256506 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:06.257548 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:06.257553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:06.258077 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:06.258268 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.258284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.259743 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.259748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.260800 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.260813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.262073 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.262079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.262312 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.263281 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:06.263308 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:06.265974 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:06.270696 (+ 4722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.270702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.270706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.271325 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.271895 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.271900 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.271903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.272087 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.273220 (+ 1133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:06.273226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:06.273403 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 21:29:06.273687 (+ 284us) client_negotiation.cc:240] Negotiation successful
0926 21:29:06.274017 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":51}
I20260926 21:29:06.275178 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.254424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.14.142.62:34719)
0926 21:29:06.255129 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.255133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.256060 (+ 927us) server_negotiation.cc:407] Connection header received
0926 21:29:06.256122 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.256126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.256200 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.256300 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.258347 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.259524 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.261756 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.262378 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.262524 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.267008 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.267013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.267016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.267046 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.271627 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.272485 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.272493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.272499 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.272675 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.273124 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.273131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.273138 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.273527 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.273670 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.274562 (+ 892us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.274779 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":457,"threads_started":1}
I20260926 21:29:06.275741 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.256329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33512 (local address 127.14.142.60:41985)
0926 21:29:06.257216 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.257221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.257247 (+ 26us) server_negotiation.cc:407] Connection header received
0926 21:29:06.257300 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.257305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.257374 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.257498 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.258436 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.259576 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.260948 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.261912 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.262096 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.266304 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.266320 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.266343 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.266387 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.270522 (+ 4135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.271491 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.271499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.271507 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.271684 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.272235 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.272240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.272242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.273022 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.273279 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.274000 (+ 721us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.274529 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":725,"spinlock_wait_cycles":1408,"thread_start_us":502,"threads_started":1}
I20260926 21:29:06.276728 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.254418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.62:34719 (local address 127.0.0.1:34234)
0926 21:29:06.255706 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.255762 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:06.255945 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:06.256673 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:06.256684 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:06.257555 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:06.258143 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.258167 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.259712 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.259717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.261603 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.261614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.262674 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.262683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.262862 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.263664 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:06.263707 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:06.266788 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:06.271774 (+ 4986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.271788 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.271798 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.272348 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.272792 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.272797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.272801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.272983 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.273747 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:06.273756 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:06.273983 (+ 227us) client_negotiation.cc:769] Sending connection context
0926 21:29:06.274197 (+ 214us) client_negotiation.cc:240] Negotiation successful
0926 21:29:06.276261 (+ 2064us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":1706,"spinlock_wait_cycles":29568,"thread_start_us":835,"threads_started":1}
I20260926 21:29:06.276888 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.251564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.14.142.60:41985)
0926 21:29:06.252401 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.252413 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.252497 (+ 84us) server_negotiation.cc:407] Connection header received
0926 21:29:06.254643 (+ 2146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.254675 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.255316 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.255747 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.258041 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.259705 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.261558 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.262132 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.262237 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.267971 (+ 5734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.267987 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.267991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.268024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.271692 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.272630 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.272635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.272638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.272770 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.273193 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.273201 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.273206 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.273527 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.273780 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.274356 (+ 576us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.276431 (+ 2075us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1972,"server-negotiator.queue_time_us":625,"thread_start_us":386,"threads_started":1}
I20260926 21:29:06.278596 5486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:06.283484 5337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: Bootstrap starting.
I20260926 21:29:06.288277 5337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: Neither blocks nor log segments found. Creating new log.
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.62@KRBTEST.COM
I20260926 21:29:06.289901 5337 log.cc:824] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: Log is configured to *not* fsync() on all Append() calls
I20260926 21:29:06.294355 5337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: No bootstrap required, opened a new log
I20260926 21:29:06.297402 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.280997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.61:36751 (local address 127.0.0.1:54176)
0926 21:29:06.281246 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.281263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:06.281394 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:06.281768 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:06.281772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:06.282220 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:06.282614 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.282650 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.284104 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.284111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.285348 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.285365 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.286102 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.286108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.286298 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.286901 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:06.286928 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:06.289830 (+ 2902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:06.294391 (+ 4561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.294398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.294403 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.295017 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.295436 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.295443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.295447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.295633 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.296302 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:06.296308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:06.296478 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 21:29:06.296691 (+ 213us) client_negotiation.cc:240] Negotiation successful
0926 21:29:06.297039 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260926 21:29:06.297446 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.281022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.14.142.61:36751)
0926 21:29:06.281209 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.281216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.281332 (+ 116us) server_negotiation.cc:407] Connection header received
0926 21:29:06.281519 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.281524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.281613 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.281716 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.282919 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.283909 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.285514 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.285945 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.286088 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.290072 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.290082 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.290089 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.290138 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.294184 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.295145 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.295151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.295153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.295316 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.295740 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.295745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.295748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.296140 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.296350 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.296815 (+ 465us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.297059 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 21:29:06.298820 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.279091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.14.142.62:34719)
0926 21:29:06.279247 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.279253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.279321 (+ 68us) server_negotiation.cc:407] Connection header received
0926 21:29:06.279540 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.279545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.279643 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.279758 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.281136 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.282154 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.283716 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.284475 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.284695 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.291245 (+ 6550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.291256 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.291260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.291307 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.294779 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.295678 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.295686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.295691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.295841 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.296235 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.296241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.296244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.296727 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.297028 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.298240 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.298448 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 21:29:06.298915 5486 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } has no permanent_uuid. Determining permanent_uuid...
I20260926 21:29:06.305399 5337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
I20260926 21:29:06.305846 5337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280ed4fc559a4f0ab1484b2d7516c2d1, State: Initialized, Role: FOLLOWER
I20260926 21:29:06.306419 5337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458146, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.60@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:29:06.307425 5494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.307802 5494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.308744 5337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: configured and running, proceeding with master startup.
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.14.142.61@KRBTEST.COM
I20260926 21:29:06.316392 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.299591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33516)
0926 21:29:06.299763 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.299782 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:06.299890 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:06.300518 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:06.300522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:06.301058 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:06.301258 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.301271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.302768 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.302773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.304083 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.304095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.304824 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.304828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.305091 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.305696 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:06.305713 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:06.308539 (+ 2826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:06.313228 (+ 4689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.313238 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.313245 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.313965 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.314444 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.314448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.314451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.314627 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.315366 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:06.315371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:06.315555 (+ 184us) client_negotiation.cc:769] Sending connection context
0926 21:29:06.315696 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 21:29:06.316000 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":20}
I20260926 21:29:06.317484 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.300091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.14.142.60:41985)
0926 21:29:06.300252 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.300258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.300274 (+ 16us) server_negotiation.cc:407] Connection header received
0926 21:29:06.300321 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.300326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.300394 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.300500 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.301411 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.302647 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.304218 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.304685 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.304815 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.308786 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.308793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.308799 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.308845 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.312723 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.314136 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.314143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.314147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.314302 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.314762 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.314766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.314769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.315243 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.316827 (+ 1584us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.316953 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.317163 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":58}
I20260926 21:29:06.319799 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.301024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.14.142.61:36751)
0926 21:29:06.301231 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.301237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.301253 (+ 16us) server_negotiation.cc:407] Connection header received
0926 21:29:06.301304 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.301311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.301393 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.301476 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.302673 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.303779 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.305588 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.306214 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.306343 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.312106 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.312111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.312115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.312151 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.315747 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.316662 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.316667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.316671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.316860 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.317325 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.317330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.317333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.317687 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.317972 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.319220 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.319454 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 21:29:06.321760 5505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:29:06.321962 5505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:29:06.323897 5486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: Bootstrap starting.
I20260926 21:29:06.326164 14904 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 21:29:06.329535 5312 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34248
I20260926 21:29:06.329789 5486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: Neither blocks nor log segments found. Creating new log.
I20260926 21:29:06.330044 5312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 21:29:06.330279 5312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:29:06.331488 5486 log.cc:824] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: Log is configured to *not* fsync() on all Append() calls
I20260926 21:29:06.334311 5312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
I20260926 21:29:06.335718 5486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: No bootstrap required, opened a new log
I20260926 21:29:06.335795 5312 leader_election.cc:288] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [CANDIDATE]: Term 1 election: Requested vote from peers cebf83bacad94cbfb0e25f208d286a50 (127.14.142.61:36751), 572317fe018d40d7b686fa5622042ef2 (127.14.142.60:41985)
W20260926 21:29:06.340747 5291 tablet.cc:2369] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:29:06.322324 5270 master_runner.cc:186] Error getting master registration for 127.14.142.61:36751: OK, instance_id { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" instance_seqno: 1790458145654627 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 21:29:06.346119 5270 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 21:29:06.347656 5486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
I20260926 21:29:06.348208 5486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572317fe018d40d7b686fa5622042ef2, State: Initialized, Role: FOLLOWER
I20260926 21:29:06.348783 5486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
I20260926 21:29:06.362047 5512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.362629 5512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.363917 5486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:29:06.363983 5461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "572317fe018d40d7b686fa5622042ef2"
I20260926 21:29:06.364656 5461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:29:06.368353 5461 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280ed4fc559a4f0ab1484b2d7516c2d1 in term 1.
I20260926 21:29:06.369127 5286 leader_election.cc:302] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280ed4fc559a4f0ab1484b2d7516c2d1, 572317fe018d40d7b686fa5622042ef2; no voters:
I20260926 21:29:06.369712 5494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 21:29:06.385921 5411 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } attempt: 5
I20260926 21:29:06.386179 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.351176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34254 (local address 127.14.142.62:34719)
0926 21:29:06.351397 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.351404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.351520 (+ 116us) server_negotiation.cc:407] Connection header received
0926 21:29:06.351871 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.351878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.351972 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.352062 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.355421 (+ 3359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.356381 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.358562 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.359387 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.359505 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.376252 (+ 16747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.376262 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.376267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.376331 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.380538 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.381471 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.381477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.381481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.381741 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.382546 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.382582 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.382588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.383326 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.383856 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.384907 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.385364 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 21:29:06.387809 5494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [term 1 LEADER]: Becoming Leader. State: Replica: 280ed4fc559a4f0ab1484b2d7516c2d1, State: Running, Role: LEADER
I20260926 21:29:06.388684 5494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
Sep 26 21:29:06 dist-test-slave-4d5t krb5kdc[5257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790458145, etypes {rep=17 tkt=17 ses=17}, kudu/127.14.142.61@KRBTEST.COM for kudu/127.14.142.60@KRBTEST.COM
I20260926 21:29:06.402057 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.351276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.14.142.61:36751)
0926 21:29:06.351570 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.351577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.361256 (+ 9679us) server_negotiation.cc:407] Connection header received
0926 21:29:06.361465 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.361473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.361581 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.361693 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.365371 (+ 3678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.366477 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.370035 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.370833 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.370949 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.384536 (+ 13587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.384545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.384572 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.384624 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.394189 (+ 9565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.396086 (+ 1897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.396092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.396096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.396286 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.397153 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.397159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.397163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.397739 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.398003 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.400205 (+ 2202us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.400474 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260926 21:29:06.402670 5517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 280ed4fc559a4f0ab1484b2d7516c2d1. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.403174 5517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: This master's current role is: LEADER
I20260926 21:29:06.404461 5489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.361479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.14.142.60:41985)
0926 21:29:06.361700 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.361707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.369360 (+ 7653us) server_negotiation.cc:407] Connection header received
0926 21:29:06.369789 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.369795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.369889 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.370000 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.372804 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.373948 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.377387 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.378133 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.378263 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.389695 (+ 11432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.389707 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.389714 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.389763 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.393996 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.396801 (+ 2805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.396810 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.396816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.396994 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.398895 (+ 1901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.398901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.398904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.399376 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.399613 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.403734 (+ 4121us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.404057 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 21:29:06.408989 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.386705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.14.142.60:41985 (local address 127.0.0.1:33544)
0926 21:29:06.387153 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 21:29:06.387178 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 21:29:06.387313 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 21:29:06.388351 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 21:29:06.388355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 21:29:06.388938 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0926 21:29:06.389154 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.389166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.390808 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.390818 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.392358 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 21:29:06.392377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.394404 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.394410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 21:29:06.394807 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.395596 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 21:29:06.395617 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 21:29:06.399613 (+ 3996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 21:29:06.405003 (+ 5390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.405008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.405032 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.405968 (+ 936us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.406694 (+ 726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 21:29:06.406699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 21:29:06.406702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 21:29:06.406980 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 21:29:06.407836 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 21:29:06.407840 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 21:29:06.408003 (+ 163us) client_negotiation.cc:769] Sending connection context
0926 21:29:06.408157 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 21:29:06.408499 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260926 21:29:06.409816 5521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 21:29:06.410332 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:06.387605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.14.142.60:41985)
0926 21:29:06.387845 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.387852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.387871 (+ 19us) server_negotiation.cc:407] Connection header received
0926 21:29:06.387933 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.387938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.388017 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.388123 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.389381 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.390574 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.392590 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:06.393383 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:06.393521 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:06.399890 (+ 6369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 21:29:06.399896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 21:29:06.399901 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 21:29:06.399939 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 21:29:06.404759 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.406174 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.406180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.406186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.406434 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 21:29:06.407177 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 21:29:06.407182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 21:29:06.407186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 21:29:06.407591 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 21:29:06.407725 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:06.408333 (+ 608us) server_negotiation.cc:299] Negotiation successful
0926 21:29:06.408736 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 21:29:06.418162 5411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: Bootstrap starting.
I20260926 21:29:06.423676 5411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: Neither blocks nor log segments found. Creating new log.
I20260926 21:29:06.426326 5521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 21:29:06.431169 5411 log.cc:824] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: Log is configured to *not* fsync() on all Append() calls
I20260926 21:29:06.443583 5411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: No bootstrap required, opened a new log
I20260926 21:29:06.434161 5419 master_runner.cc:186] Error getting master registration for 127.14.142.61:36751: OK, instance_id { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" instance_seqno: 1790458145654627 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 21:29:06.446173 5419 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 21:29:06.448086 5461 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [term 1 FOLLOWER]: Refusing update from remote peer 280ed4fc559a4f0ab1484b2d7516c2d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:29:06.449911 5517 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 21:29:06.451006 5529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 21:29:06.451134 5529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 21:29:06.461054 5411 raft_consensus.cc:348] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
I20260926 21:29:06.461654 5411 raft_consensus.cc:729] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cebf83bacad94cbfb0e25f208d286a50, State: Initialized, Role: FOLLOWER
I20260926 21:29:06.462284 5411 consensus_queue.cc:260] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } }
W20260926 21:29:06.470492 5440 tablet.cc:2369] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 21:29:06.471082 5512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 280ed4fc559a4f0ab1484b2d7516c2d1. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.471441 5512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.478328 5411 sys_catalog.cc:564] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 21:29:06.478369 5386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cebf83bacad94cbfb0e25f208d286a50"
I20260926 21:29:06.478789 5385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [term 0 FOLLOWER]: Advancing to term 1
I20260926 21:29:06.482688 5385 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [term 1 FOLLOWER]: Refusing update from remote peer 280ed4fc559a4f0ab1484b2d7516c2d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 21:29:06.486413 5386 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280ed4fc559a4f0ab1484b2d7516c2d1 in term 1.
I20260926 21:29:06.488670 5494 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 21:29:06.477807 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.516142 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.525767 5494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.526325 5494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: This master's current role is: LEADER
I20260926 21:29:06.537979 5521 catalog_manager.cc:1347] Generated new cluster ID: 6a5ce6ff62fe4655a988402d51f50646
I20260926 21:29:06.538122 5521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 21:29:06.539949 5512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.540217 5512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.553045 5539 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 280ed4fc559a4f0ab1484b2d7516c2d1. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.553514 5539 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.553982 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.554268 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 21:29:06.573774 5517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280ed4fc559a4f0ab1484b2d7516c2d1" member_type: VOTER last_known_addr { host: "127.14.142.62" port: 34719 } } peers { permanent_uuid: "cebf83bacad94cbfb0e25f208d286a50" member_type: VOTER last_known_addr { host: "127.14.142.61" port: 36751 } } peers { permanent_uuid: "572317fe018d40d7b686fa5622042ef2" member_type: VOTER last_known_addr { host: "127.14.142.60" port: 41985 } } }
I20260926 21:29:06.574164 5517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1 [sys.catalog]: This master's current role is: LEADER
I20260926 21:29:06.584149 5521 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 21:29:06.586822 5521 catalog_manager.cc:1504] Loading token signing keys...
I20260926 21:29:06.610942 5521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 280ed4fc559a4f0ab1484b2d7516c2d1: Generated new TSK 0
I20260926 21:29:06.612433 5521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 21:29:06.639639 5553 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 21:29:06.727173 5490 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.713534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.14.142.62:34719)
0926 21:29:06.713739 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.713745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.725237 (+ 11492us) server_negotiation.cc:407] Connection header received
0926 21:29:06.725550 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.725556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.725646 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.725727 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.726937 (+ 1210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34268: BlockingRecv error: recv got EOF from 127.0.0.1:34268 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260926 21:29:06.729343 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.713460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.14.142.61:36751)
0926 21:29:06.713695 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.713703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.727334 (+ 13631us) server_negotiation.cc:407] Connection header received
0926 21:29:06.727740 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.727756 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.727847 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.727929 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.729098 (+ 1169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54194: BlockingRecv error: recv got EOF from 127.0.0.1:54194 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260926 21:29:06.729537 5492 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.725573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.14.142.60:41985)
0926 21:29:06.725781 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.725787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.727622 (+ 1835us) server_negotiation.cc:407] Connection header received
0926 21:29:06.728028 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.728036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.728122 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.728191 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.729279 (+ 1088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33554: BlockingRecv error: recv got EOF from 127.0.0.1:33554 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 21:29:06.736123 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.734091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.14.142.61:36751)
0926 21:29:06.734302 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.734312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.734746 (+ 434us) server_negotiation.cc:407] Connection header received
0926 21:29:06.734981 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.734985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.735075 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.735193 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.735875 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54210: BlockingRecv error: recv got EOF from 127.0.0.1:54210 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 21:29:06.742357 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.740251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54226 (local address 127.14.142.61:36751)
0926 21:29:06.740477 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.740486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.740993 (+ 507us) server_negotiation.cc:407] Connection header received
0926 21:29:06.741197 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.741204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.741286 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.741397 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.742128 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54226: BlockingRecv error: recv got EOF from 127.0.0.1:54226 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 21:29:06.757465 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.754173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54240 (local address 127.14.142.61:36751)
0926 21:29:06.754396 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.754405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.755909 (+ 1504us) server_negotiation.cc:407] Connection header received
0926 21:29:06.756142 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.756146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.756232 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.756353 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.757189 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54240: BlockingRecv error: recv got EOF from 127.0.0.1:54240 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 21:29:06.768431 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.765833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.14.142.61:36751)
0926 21:29:06.766112 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.766119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.766773 (+ 654us) server_negotiation.cc:407] Connection header received
0926 21:29:06.767089 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.767095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.767204 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.767337 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.768162 (+ 825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54244: BlockingRecv error: recv got EOF from 127.0.0.1:54244 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
W20260926 21:29:06.782790 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.780500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.14.142.61:36751)
0926 21:29:06.780780 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.780790 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.781368 (+ 578us) server_negotiation.cc:407] Connection header received
0926 21:29:06.781602 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.781609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.781694 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.781816 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.782535 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54252: BlockingRecv error: recv got EOF from 127.0.0.1:54252 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
W20260926 21:29:06.813813 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.810640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.14.142.61:36751)
0926 21:29:06.810898 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.810909 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.811825 (+ 916us) server_negotiation.cc:407] Connection header received
0926 21:29:06.812452 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.812460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.812631 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.812761 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.813566 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54260: BlockingRecv error: recv got EOF from 127.0.0.1:54260 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260926 21:29:06.853168 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.850833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54264 (local address 127.14.142.61:36751)
0926 21:29:06.851066 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.851075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.851622 (+ 547us) server_negotiation.cc:407] Connection header received
0926 21:29:06.851869 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.851874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.851956 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.852069 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.852882 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54264: BlockingRecv error: recv got EOF from 127.0.0.1:54264 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260926 21:29:06.928376 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:06.925868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.14.142.61:36751)
0926 21:29:06.926076 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:06.926083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:06.926711 (+ 628us) server_negotiation.cc:407] Connection header received
0926 21:29:06.926957 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:06.926964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:06.927066 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:06.927197 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:06.928121 (+ 924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54278: BlockingRecv error: recv got EOF from 127.0.0.1:54278 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 21:29:07.077498 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:07.074692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.14.142.61:36751)
0926 21:29:07.075119 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.075128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.075330 (+ 202us) server_negotiation.cc:407] Connection header received
0926 21:29:07.075685 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.075694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.075819 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.075940 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:07.077188 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54286: BlockingRecv error: recv got EOF from 127.0.0.1:54286 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
W20260926 21:29:07.339669 5416 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 21:29:07.337628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.14.142.61:36751)
0926 21:29:07.337899 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.337906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.338048 (+ 142us) server_negotiation.cc:407] Connection header received
0926 21:29:07.338332 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.338337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.338427 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.338553 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0926 21:29:07.339393 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54298: BlockingRecv error: recv got EOF from 127.0.0.1:54298 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
I20260926 21:29:07.453576 5529 catalog_manager.cc:1259] Loaded cluster ID: 6a5ce6ff62fe4655a988402d51f50646
I20260926 21:29:07.453696 5529 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: loading cluster ID for follower catalog manager: success
I20260926 21:29:07.456331 5529 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: acquiring CA information for follower catalog manager: success
I20260926 21:29:07.458310 5529 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 572317fe018d40d7b686fa5622042ef2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:29:07.644414 5553 catalog_manager.cc:1259] Loaded cluster ID: 6a5ce6ff62fe4655a988402d51f50646
I20260926 21:29:07.644686 5553 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: loading cluster ID for follower catalog manager: success
I20260926 21:29:07.647609 5553 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: acquiring CA information for follower catalog manager: success
I20260926 21:29:07.650128 5553 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cebf83bacad94cbfb0e25f208d286a50: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 21:29:07.865329 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:07.857363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.14.142.61:36751)
0926 21:29:07.858323 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.858334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.858950 (+ 616us) server_negotiation.cc:407] Connection header received
0926 21:29:07.859229 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.859233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.859389 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.859563 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:07.860078 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.861139 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.862649 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.863143 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.863313 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:07.863893 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:07.864606 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:07.864790 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:07.864848 (+ 58us) server_negotiation.cc:299] Negotiation successful
0926 21:29:07.864943 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":456,"threads_started":1}
I20260926 21:29:07.883471 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:07.873306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.14.142.60:41985)
0926 21:29:07.873832 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.873837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.877224 (+ 3387us) server_negotiation.cc:407] Connection header received
0926 21:29:07.877513 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.877518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.877704 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.877911 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:07.878482 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.879392 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.880965 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.881401 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.881555 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:07.882171 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:07.882769 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:07.882955 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:07.883023 (+ 68us) server_negotiation.cc:299] Negotiation successful
0926 21:29:07.883128 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":301,"threads_started":1}
I20260926 21:29:07.898962 5632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:07.891146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.14.142.62:34719)
0926 21:29:07.891855 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.891861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.891882 (+ 21us) server_negotiation.cc:407] Connection header received
0926 21:29:07.892153 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.892158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.892333 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.892536 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:07.893057 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.893847 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.895893 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.896453 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.896660 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:07.897345 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:07.897817 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:07.897956 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:07.898233 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 21:29:07.898361 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":463,"threads_started":1}
I20260926 21:29:07.904417 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 5270
I20260926 21:29:07.905539 5332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:29:07.905946 5619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:07.890854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.14.142.61:36751)
0926 21:29:07.891056 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.891066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.898795 (+ 7729us) server_negotiation.cc:407] Connection header received
0926 21:29:07.899388 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.899394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.899504 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.899578 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:07.900247 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.901424 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.903476 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.904200 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.904326 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:07.905119 (+ 793us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:07.905270 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:07.905332 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:07.905535 (+ 203us) server_negotiation.cc:299] Negotiation successful
0926 21:29:07.905618 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:29:07.909089 5625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 21:29:07.901388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.14.142.60:41985)
0926 21:29:07.901583 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 21:29:07.901590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 21:29:07.901799 (+ 209us) server_negotiation.cc:407] Connection header received
0926 21:29:07.901879 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 21:29:07.901886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 21:29:07.901989 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 21:29:07.902076 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 21:29:07.903255 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.904353 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.906728 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 21:29:07.907355 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 21:29:07.907474 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 21:29:07.908133 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 21:29:07.908302 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 21:29:07.908353 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0926 21:29:07.908681 (+ 328us) server_negotiation.cc:299] Negotiation successful
0926 21:29:07.908780 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 21:29:08.045549 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 5270
I20260926 21:29:08.068534 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 5344
I20260926 21:29:08.069736 5406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:29:08.181514 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 5344
I20260926 21:29:08.202939 14904 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu pid 5419
I20260926 21:29:08.204645 5481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 21:29:08.315088 14904 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh_JUDQ/build/asan/bin/kudu with pid 5419
2026-09-26T21:29:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5147 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10204 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (386320 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST